Method and apparatus for training of cognitive and memory systems in humans

ABSTRACT

An apparatus and method for training the cognitive and memory systems in a subject is provided. The apparatus and method incorporates a number of different programs to be played by the subject. The programs artificially process selected portions of language elements, called phonemes, so they will be more easily distinguished by the subject, and gradually improves the subject&#39;s neurological processing and memory of the elements through repetitive stimulation. The programs continually monitor a subject&#39;s ability to distinguish the processed language elements, and adaptively configures the programs to challenge and reward the subject by altering the degree of processing. Through adaptive control and repetition of processed speech elements, and presentation of the speech elements in a creative fashion, a subject&#39;s cognitive processing of acoustic events common to speech, and memory of language constructs associated with speech elements are significantly improved.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Divisional application of co-pending U.S. patentapplication Ser. No. 08/992,072, filed Dec. 17, 1997 entitled “METHODAND APPARATUS FOR TRAINING OF COGNITIVE AND MEMORY SYSTEMS IN HUMANS.”This application is related to co-pending U.S. patent application Ser.No. 08/982,189 entitled “METHOD AND APPARATUS FOR TRAINING OF SENSORYAND PERCEPTUAL SYSTEMS IN LLI SUBJECTS”; and U.S. patent applicationSer. No. 08/992,071, filed Dec. 17, 1997 entitled “METHOD AND APPARATUSFOR TRAINING OF SENSORY AND PERCEPTUAL SYSTEMS IN LLI SUBJECTS”; bothassigned to Scientific Learning Corporation, and incorporated herein byreference for all purposes.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates in general to the field of education in humansubjects, and more specifically to a computer program for training thecognitive and memory processing systems in human subjects.

2. Description of the Related Art

Up to ten percent of humans have some type of language-learningimpairments (LLI) resulting from the inability to accurately processshort duration acoustic events at the rates that occur in normal speech.Their trouble distinguishing among elements of speech is neurologicallybased and has far reaching consequences: academic failure, emotional anddisciplinary problems, and possibly diminished lifelong achievement andself-image. No bracket of intelligence, race, gender or economic levelis immune from this problem.

More specifically, people with LLI have difficulty detecting andidentifying sounds that occur simultaneously or in close proximity toeach other—a phenomenon known as “masking.” Because of masking, peoplewith LLI require sounds that are as much as 45 decibels more intensethan the preceding or subsequent masking noise to distinguish andunderstand them. In addition, people with LLI are consistently poorer atdetecting a brief tone presented with a masking noise, particularly whenthe brief tone is turned on immediately prior to the masking noise. Thisphenomenon is called “backward masking.” Similarly, when the brief toneis turned on immediately after the masking noise a similar decrease indetectability can occur. This phenomenon is called “forward masking”.For a tone to be detected by a person with LLI in the presence of amasking noise, the tone must be separated in time or frequency from themasking noise.

One way individuals develop such auditory processing problems is frommiddle ear infections when they are young and beginning to develop theoral representations of language in the central auditory nervous system.When a child has an ear infection, fluid can build up and block ormuffle the sound wave entering the ear causing intermittent hearingloss. Even if the infection doesn't permanently damage the ear, thechild's brain doesn't learn to process some sounds because it hasn'theard them accurately before, on a consistent basis. This typicallyoccurs during a critical period of brain development when the brain isbuilding the nerve connections necessary to accurately process acousticevents associated with normal speech. As the child grows up, the nervoussystem never builds, or rebuilds the connections necessary to properlyprocess normal speech.

Researchers believe that the auditory processing problem is essentiallyone of timing. Vowel sounds like /a/ and /e/ usually last at least 100milliseconds and typically have constant frequency content. Consonants,on the other hand, typically have modulated frequency components, andlast less than 40 milliseconds. People with LLI cannot process thesefaster speech elements, especially the hard consonants like /t/, /p/,/d/ and /b/, if they occur either immediately before or after vowels, orif they are located near other consonants. Rather than hearing theindividual sounds that make up a particular phoneme, people with LLIintegrate closely associated sounds together over time. Since theduration of vowels are typically longer than consonants, the modulatedfrequency portions of consonants are often lost in the integration, anaffect that may also hinder the resolution of the vowel, particularlyshort duration vowels.

This problem of abnormal temporal integration of acoustic events overtime is not limited to people with LLI. Rather, the problem extends tostroke victims who have lost the neurological connections necessary toprocess speech, as well as to individuals raised in one country, havingone set of language phonemes, and attempting to learn the language ofanother country, having a distinct set of language phonemes. Forexample, it is known that an individual raised in Japan is not oftenpresented with phonemes similar to the English r's and l's, becausethose consonants are not common in the Japanese language. Similarly,there are many subtleties in the sounds made by a speaker of Japanesethat are difficult to distinguish unless raised in Japan. The phoneticdifferences between languages are distinctions that must be learned, andare often very difficult. But, they are clearly problems that relate tothe temporal processing of short duration acoustic events.

The above described temporal processing deficiency has little ifanything to do with intelligence. In fact, some LLI specialists arguethat brains choosing this different route by which to absorb andreassemble bits of speech may actually stimulate creative intelligence,but at the expense of speech and reading problems.

Recent studies have shown that if the acoustic events associated withphonemes that are difficult to distinguish, such as /ba/ and /da/, areslowed down, or that the consonant portion of the phonemes areemphasized, that students diagnosed as LLI can accurately distinguishbetween the phonemes. In addition, if the interval between two complexsounds is lengthened, LLI students are better able to process the soundsdistinctly.

Heretofore, the solution to the processing problem has been to place LLIstudents in extended special education and/or speech therapy trainingprograms that focus on speech recognition and speech production. Or,more commonly, repetitive reading programs, phonic games, or otherphonic programs are undertaken. These programs often last for years,with a success rate that is often more closely associated with the skillof the speech and language professional than with the program of study.

In addition to the temporal processing problems discussed above, manyindividuals have a decreased ability to remember and process logicalconstructs that are common in language. For example, a simple commandsuch as “place the small red circle inside the large green square”,requires an individual to logically associate the adjectives “small” and“red” with the subject “circle”, and the adjectives “large” and “green”with the subject “square”. The command further requires that theindividual manipulate the first subject “circle” by placing it into thesecond subject “square”. Individuals with abnormal cognitive processing,or with abnormal short term memory, become easily confused whenpresented with such commands.

What is needed is a method and apparatus that allows a subject withabnormal temporal processing to train, or retrain their brain torecognize and distinguish short duration acoustic events that are commonin speech. Moreover, what is needed is a program that repetitivelytrains a subject to distinguish phonemes at a normal rate, by firststretching, and/or emphasizing elements of speech to the point that theyare distinguishable, or separating speech elements in time, and thenadaptively adjusting the stretching, emphasis and separation of thespeech elements to the level of normal speech. The adaptive adjustmentsshould be made so as to encourage the subject to continue with therepetitions, and the number of repetitions should be sufficient todevelop the necessary neurological connections for normal temporalprocessing of speech. Moreover, the program should provide acousticsignals to the brain that are better for phonetic training than normalhuman speech.

Furthermore, what is needed is an adaptive training method to trainindividuals to accurately process logical constructs that are common inlanguage. In addition, what is needed is a method to improve anindividual's short term memory to allow them to correctly process bothsimple and complex language constructs.

SUMMARY

To address the above-detailed deficiencies, the present inventionprovides a method to improve a subject's comprehension of logicalconstructs within language using modified speech commands. The methodprovides a plurality of training skill levels for the speech commands,the plurality of training skill levels differing from each other in thedifficulty of logical constructs, and in the amount of audio processingapplied to the speech commands. The method also selects from theplurality of training skill levels, a training skill level to bepresented to the subject that is associated with, or corresponds to, thesubject's ability. The method also presents a game board that has aplurality of graphical components, the plurality of graphical componentsbeing the subject of the modified speech commands. The method furtherpresents as a trial, a speech command from a set of speech commands fromthe selected skill level. The speech command directs the subject tomanipulate at least one of the plurality of graphical components. If thesubject correctly manipulates the graphical components, the methodpresents another trial. If the subject incorrectly manipulates thegraphical components, the method visually indicates to the subject thecorrect manipulation. As the subject correctly manipulates the graphicalcomponents, the amount of processing applied to the speech commands isreduced. And, as the subject incorrectly manipulates the graphicalcomponents, the amount of processing applied to the speech commands isincreased. Thus, the method adaptively trains the subject to correctlydistinguish between similar speech commands.

In another aspect, the present invention provides a method to improvethe cognitive processing system of a subject. The method is presented tothe subject on a computer, and interacts with the subject viainput/output devices on the computer. The method provides a plurality ofstimulus sets, each of the plurality of stimulus sets having a pluralityof command sentences. The method also provides a plurality of targetgraphical images, each of the graphical images associated with adifferent one of the plurality of command sentences. The method furtherprovides a plurality of distractor images that are not associated withthe plurality of command sentences. The method then presents to thesubject one of the plurality of command sentences from one of theplurality of stimulus sets to the subject, the presented sentencemodified acoustically, and presents to the subject a target graphicalimage, from the plurality of target graphical images, that is associatedwith the presented command sentence. Along with the presented targetgraphical image, the method presents a plurality of distractor images.The subject is then required to distinguish between the presented targetgraphical image, and the presented plurality of distractor images byselecting the target graphical image associated with the presentedcommand sentence.

In yet another aspect, the present invention provides an adaptive methodto improve a subject's short term memory. The method utilizes a computerto process and present sounds to the subject. The method displays aplurality of graphical images on the computer, the graphical imagesassociated with acoustic events. The method associates in pairs theplurality of graphical images with particular acoustic events such thattwo different graphical images are associated with a particular acousticevent. Upon the subject's selection of any of the plurality of graphicalimages, its associated acoustic event is presented. The method thenrequires the user to discriminate between the acoustic events bysequentially selecting two different graphical images from among theplurality of graphical images, that are associated with the particularacoustic event. The acoustic events are modified by stretching them inthe time domain by varying amounts. As the subject correctly remembersthe acoustic events at one skill level, the amount of stretching appliedto the acoustic events is reduced. In addition, as the subject correctlyremembers the acoustic events, the number of graphical image pairspresented to the subject increases, requiring the subject to bettertrain his/her memory.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objects, features, and advantages of the presentinvention will become better understood with regard to the followingdescription, and accompanying drawings where:

FIG. 1 is a block diagram of a computer system for executing a programaccording to the present invention.

FIG. 2 is a block diagram of a computer network for executing a programaccording to the present invention.

FIG. 3 is a chart illustrating frequency/energy characteristics of twophonemes within the English language.

FIG. 4 is a chart illustrating auditory reception of a phoneme by asubject having normal receptive characteristics, and by a subject whosereceptive processing is impaired.

FIG. 5 is a chart illustrating stretching of a frequency envelope intime, according to the present invention.

FIG. 6 is a chart illustrating emphasis of selected frequencycomponents, according to the present invention.

FIG. 7 is a chart illustrating up-down frequency sweeps of varyingduration, separated by a selectable inter-stimulus-interval (ISI),according to the present invention.

FIG. 8 is a pictorial representation of a game selection screenaccording to the present invention.

FIG. 9 is a pictorial representation of a game entitled “Old MacDonald'sFlying Farm” according to the present invention.

FIG. 10 is a flow chart illustrating the adaptive auditory trainingprocedures embodied in the game Old MacDonald's Flying Farm.

FIGS. 11 and 12 are pictorial representations of a game entitled “BlockCommander” according to the present invention.

FIG. 13 is a flow chart illustrating the adaptive auditory trainingprocedures embodied in the game Block Commander.

FIGS. 14 and 15 are pictorial representations of a game entitled “CircusSequence” according to the present invention.

FIG. 16 is a flow chart illustrating the initial training proceduresembodied in the game Circus Sequence.

FIG. 17 is a flow chart illustrating the adaptive auditory trainingprocedures embodied in the game Circus Sequence.

FIG. 18 is a pictorial representation of a game entitled “Phonic Match”according to the present invention.

FIG. 19 includes two tables illustrating the processing levels and thetraining levels embodied in the game Phonic Match.

FIG. 20 is a flow chart illustrating the adaptive auditory trainingprocess embodied in the game Phonic Match.

FIGS. 21 and 22 are pictorial representations of a game entitled “PhonicWords” according to the present invention.

FIG. 23 is a flow chart illustrating the adaptive auditory trainingprocess embodied in the game Phonic Words.

FIGS. 24 and 25 are pictorial representations of a game entitled“Phoneme Identification” according to the present invention.

FIG. 26 is a flow chart illustrating the initial training proceduresembodied in the game Phoneme Identification.

FIG. 27 is a flow chart illustrating the adaptive auditory trainingprocess embodied in the game Phoneme Identification.

FIG. 28 is a pictorial representation of a game entitled “LanguageComprehension Builder” according to the present invention.

FIG. 29 is a flow chart illustrating the initial training proceduresembodied in the game Language Comprehension Builder.

FIG. 30 is a flow chart illustrating the adaptive auditory trainingprocedures embodied in the game Language Comprehension Builder.

FIG. 31 is a flow chart illustrating a time-scale modification algorithmfor modifying acoustic elements according to the present invention.

FIG. 32 is a flow chart illustrating a filter-bank summation emphasisalgorithm for modifying acoustic elements according to the presentinvention.

FIG. 33 is a flow chart illustrating an overlap-add emphasis algorithmfor modifying acoustic elements according to the present invention.

DETAILED DESCRIPTION

Referring to FIG. 1, a computer system 100 is shown for executing acomputer program to train, or retrain a learning language impaired (LLI)subject, according to the present invention. The computer system 100contains a computer 102, having a CPU, memory, hard disk and CD ROMdrive (not shown), attached to a monitor 104. The monitor 104 providesvisual prompting and feedback to the subject during execution of thecomputer program. Attached to the computer 102 are a keyboard 105,speakers 106, a mouse 108, and headphones 110. The speakers 106 and theheadphones 110 provide auditory prompting and feedback to the subjectduring execution of the computer program. The mouse 108 allows thesubject to navigate through the computer program, and to selectparticular responses after visual or auditory prompting by the computerprogram. The keyboard 105 allows an instructor to enter alpha numericinformation about the subject into the computer 102. Although a numberof different computer platforms are applicable to the present invention,embodiments of the present invention execute on either IBM compatiblecomputers or Macintosh computers.

Now referring to FIG. 2, a computer network 200 is shown. The computernetwork 200 contains computers 202, 204, similar to that described abovewith reference to FIG. 1, connected to a server 206. The connectionbetween the computers 202, 204 and the server 206 can be made via alocal area network (LAN), a wide area network (WAN), or via modemconnections, directly or through the Internet. A printer 208 is shownconnected to the computer 202 to illustrate that a subject can print outreports associated with the computer program of the present invention.The computer network 200 allows information such as test scores, gamestatistics, and other subject information to flow from a subject'scomputer 202, 204 to a server 206. An administrator can then review theinformation and can then download configuration and control informationpertaining to a particular subject, back to the subject's computer 202,204. Details of the type of information passed between a subject'scomputer and a server are provided in co-pending U.S. application Ser.No. 08/995,680, entitled “Remote Computer-Assisted ProfessionallySupervised Teaching System”, assigned to Scientific LearningCorporation.

Before providing a detailed description of the present invention, abrief overview of certain components of speech will be provided, alongwith an explanation of how these components are processed by LLIsubjects. Following the overview, general information on speechprocessing will be provided so that the reader will better appreciatethe novel aspects of the present invention.

Referring to FIG. 3, a chart is shown that illustrates frequencycomponents, over time, for two distinct phonemes within the Englishlanguage. Although different phoneme combinations are applicable toillustrate features of the present invention, the phonemes /daa/ and/ba/ are shown. For the phoneme /da/, a downward sweep frequencycomponent 302, at approximately 2.5-2 khz is shown to occur over a 35msinterval. In addition, a downward sweep frequency component 304, atapproximately 1 khz is shown to occur during the same 35 ms interval. Atthe end of the 35 ms interval, a constant frequency component 306 isshown, whose duration is approximately 110 ms. Thus, in producing thephoneme /da/, the stop consonant portion of the element /d/ isgenerated, having high frequency sweeps of short duration, followed by along vowel element /a/ of constant frequency.

Also shown are frequency components for a phoneme /ba/. This phonemecontains an upward sweep frequency component 308, at approximately 2khz, having a duration of approximately 35 ms. The phoneme also containsan upward sweep frequency component 310, at approximately 1 khz, duringthe same 35 ms period. Following the stop consonant portion /b/ of thephoneme, is a constant frequency vowel portion 314 whose duration isapproximately 110 ms.

Thus, both the /ba/ and /da/ phonemes begin with stop consonants havingmodulated frequency components of relatively short duration, followed bya constant frequency vowel component of longer duration. The distinctionbetween the phonemes exist primarily in the 2 khz sweeps during theinitial 35 ms interval. Similarity exists between other stop consonantssuch as /ta/, /pa/, /ka/ and /ga/.

Referring now to FIG. 4, the amplitude of a phoneme, for example /ba/,is viewed in the time domain. A short duration high amplitude peakwaveform 402 is created upon release of either the lips or the tonguewhen speaking the consonant portion of the phoneme, that rapidlydeclines to a constant amplitude signal of longer duration. For anindividual with normal temporal processing, the waveform 402 will beunderstood and processed essentially as it is. However, for anindividual who is learning-language impaired, or who has abnormaltemporal processing, the short duration, higher frequency consonantburst will be integrated over time with the lower frequency vowel, anddepending on the degree of impairment, will be heard as the waveform404. The result is that the information contained in the higherfrequency sweeps associated with consonant differences, will be muddled,or indistinguishable.

With the above general background of speech elements, and how LLIsubjects process them, a general overview of speech processing will nowbe provided. As mentioned above, one problem that exists in LLI subjectsis the inability to distinguish between short duration acoustic events.If the duration of these acoustic events are stretched, in the timedomain, it is possible to train LLI subjects to distinguish betweenthese acoustic events. An example of such time domain stretching isshown in FIG. 5, to which attention is now directed.

In FIG. 5, a frequency vs. time graph 500 is shown that illustrates awaveform 502 having short duration characteristics similar to thewaveform 402 described above. Using existing computer technology, theanalog waveform 502 can be sampled and converted into digital values(using a Fast Fourier Transform, for example). The values can then bemanipulated so as to stretch the waveform in the time domain to apredetermined length, while preserving the amplitude and frequencycomponents of the modified waveform. The modified waveform can then beconverted back into an analog waveform (using an inverse FFT) forreproduction by a computer, or by some other audio device. The waveform502 is shown stretched in the time domain to durations of 60 ms(waveform 504), and 80 ms (waveform 506). By stretching the consonantportion of the waveform 502 without effecting its frequency components,subjects with LLI can begin to hear distinctions in common phonemes.

Another method that may be used to help LLI subjects distinguish betweenphonemes is to emphasize selected frequency envelopes within a phoneme.Referring to FIG. 6, a graph 600 is shown illustrating a frequencyenvelope 602 whose envelope varies by approximately 27 hz. By detectingfrequency modulated envelopes that vary from say 3-30 hz, similar tofrequency variations in the consonant portion of phonemes, andselectively emphasizing those envelopes, they are made more easilydetectable by LLI subjects. A 10 dB emphasis of the envelope 602 isshown in waveform 604, and a 20 dB emphasis in the waveform 606.

A third method that may be used to train LLI subjects to distinguishshort duration acoustic events is to provide frequency sweeps of varyingduration, separated by a predetermined interval, as shown in FIG. 7.More specifically, an upward frequency sweep 702, and a downwardfrequency sweep 704 are shown, having duration's varying between 25 and80 milliseconds, and separated by an inter-stimulus interval (ISI) ofbetween 500 and 0 milliseconds. The duration and frequency of thesweeps, and the inter-stimulus interval between the sweeps are varieddepending on the processing level of the LLI subject, as will be furtherdescribed below.

Utilization of up-down frequency sweeps with varying ISI has been fullydescribed in U.S. patent application Ser. No. 08/351,803, entitled“METHOD AND DEVICE FOR ENHANCING THE RECOGNITION OF SPEECH AMONGSPEECH-IMPAIRED INDIVIDUALS”, and is hereby incorporated by reference.

Each of the above described methods have been combined in a uniquefashion by the present invention to provide an adaptive training methodand apparatus for training subjects having abnormal temporal processingabilities to recognize and distinguish short duration acoustic eventsthat are common in speech. The present invention is embodied into acomputer program entitled Fast ForWord by Scientific LearningCorporation. The computer program is provided to an LLI subject via aCD-ROM which is input into a general purpose computer such as thatdescribed above with reference to FIG. 1. In addition, a user may logonto a server, via an Internet connection, for example, to upload testresults, and to download training parameters for future exercises.Specifics of the present invention will now be described with referenceto FIGS. 8-30.

Referring first to FIG. 8, a pictorial representation is shown of a gameselection screen 800. The game selection screen 800 is similar to thatprovided to an LLI subject upon initialization of the computer programaccording to the present invention. The game selection screen 800includes the titles of seven computer games that provide distincttraining exercises for improving speech recognition in subjects whoabnormally process temporal acoustic events, and for building, orrebuilding the neurological connections necessary to accurately processphonemes at the rates common in speech. The game titles include: 1) OldMacDonald's Flying Farm; 2) Block Commander; 3) Circus Sequence; 4)Phonic Match; 5) Phonic Words; 6) Phoneme Identification; and 7)Language Comprehension Builder. Each of these games will be discussed ingreater detail below.

When a subject begins execution of the Fast ForWord computer program,he/she is presented with a screen similar to the screen 800. Morespecifically, upon initiation of the program, the subject is presentedwith a screen that lists the subjects that are currently being trainedby the program. The subject then selects his/her name from the list.Once the subject has selected his/her name, a screen similar to 800appears, typically listing one of the seven programs, according to atraining schedule that is dictated by the program, or is modified by aninstructor. The order of the games, and the selection of which one ofthe seven games that is presented in the screen 800 varies from day today. The subject then elects to play the first game listed according tothe training schedule prescribed for the subject.

In one embodiment, a training schedule is provided by a certified Speechand Language Professional (SLP), and the SLP oversees each trainingsession according to the schedule. An exemplary schedule requires asubject to cycle through five of the seven games for an hour and fortyminutes, five days per week, for approximately six weeks. In addition,the schedule typically requires that a subject play Circus Sequence andLanguage Comprehension Builder everyday, alternating the other games sothat they are played approximately the same amount of time.

In an alternative embodiment, the game schedule specified by an SLP at aremote server, and the daily parameters of the schedule are downloadedto the subject's computer, either daily or weekly. The schedule can beoptimized over the course of the training program to first developskills required for subsequent more advanced skills. It can also be usedto help manage time in each game so that all of the games are completedat about the same time at the end of the training program. Thisembodiment allows a subject to obtain the benefits of the Fast ForWordprogram, and the oversight of a certified SLP, regardless of his/hergeographic location. One skilled in the art will appreciate that thetraining schedule could either be provided in a window on the subject'scomputer, or could actually control the game selection screen to promptthe user only for those games required on a particular day.

Once a subject selects a particular game, he/she is taken into thatparticular game's module. Alternatively, once the subject selectshis/her name from the list, the particular games may be presented, in apredefined order, without requiring the subject to first select thegame. For ease of illustration, each of the seven games will bediscussed, in the order represented in FIG. 8.

Referring to FIG. 9, a scene 900 is shown for the first game in theprogram, Old MacDonald's Flying Farm (OMDFF). OMDFF uses apsychophysical procedure called limited-hold reaction time. A subject isasked to start a trial, in this case by grabbing a flying animal, atwhich point the game begins presenting a distractor phoneme that ismodified in the time domain only. More specifically, information bearingacoustic elements whose temporal location within a phoneme carryimportant cues for phoneme identification are modified by stretching theacoustic elements in time, say to 150% of their normal duration. Theacoustic elements that are stretched include voice onset time (VOT)between consonant and vowel events, as well as fricative-vowel gaps. Theinter-stimulus interval (ISI) between presentations of the distractorphoneme is set initially to 500 ms. The distractor phoneme is repeated arandom number of times, usually between 3 and 8 times, before the targettone is presented. The target phoneme has normal temporal acousticparameters. The subject is asked to continue to hold the animal untilthe target phoneme is presented. When the subject hears the targetphoneme, the subject is to release the animal. If the subject accuratelyhears the target phoneme and releases the animal within a desired “hit”window, then his/her score increases. If the subject misses the targetphoneme, the animal flies away and no points are given. As the subjectimproves, the temporal parameters of the distractor phonemes are reducedin time to that of normal speech, and the ISI is reduced, systematicallyto 300 ms.

A number of scenes are provided in OMDFF, each correlated to a specificpair of sounds. The correlation of sound pairs to farm scenes is shownbelow:

Sound Pair Scene /Gi/ - /Ki/ Barn /Chu/ - /Shu/ Mudpit /Si/ - /Sti/Garden /Ge/ - /Ke/ House /Do/ - /To/ Coop

So, when a subject grabs the flying animal, the game begins presenting atone pattern such as: /Si/ . . . /Si/ /Si/ . . . /Si/ . . . /Sti/. Whenthe subject hears /Sti/, the subject is to release the animal.

The scene 900 provides a general farmyard background with three elementsthat persist across all the scenes. The elements are the score digits906, the stop sign 908, and the tractor 910. The tractor 910 acts as aprogress creature to graphically indicate to a subject their progressduring a game. If the subject gets a correct response, the tractor 910advances across the screen 900, from right to left. The score digits 906display the subject's current score. The stop sign 908 is common to allseven games, and provides a subject with a means for exiting the game,and then the program.

Also shown on the screen 900 are a flying farm animal 902, and aselection hand 904. In this scene, the flying farm animal 902 is a cowwith a rocket pack. Other scenes provide different farm animalspropelled through the air with different flying apparatus. Operation ofthe game OMDFF will now be described with reference to FIG. 10.

In FIG. 10, a flow chart 1000 is provided that illustrates operation ofthe OMDFF game. The game begins at block 1002 and proceeds to block1004.

At block 1004, the computer program selects a particular tone sequenceto be played for a subject. For example, the program would select thetone pair /Si/ . . . /Sti/, stretched 150%, with an ISI of 500 ms. Thetone pair that is selected, the stretching, and the ISI, are allassociated with a particular skill level. And, the skill level that ispresented to a subject is adapted in real time, based on the subjectsability to recognize the target phoneme, as will be further describedbelow. However, the initial phoneme pair, stretching and ISI are chosento allow an LLI subject to understand the game, and to begin todistinguish phonemes common in speech. Upon selection of a particularphoneme sequence, and skill level, flow proceeds to block 1006.

At block 1006, the game presents a flying animal 902. As mentionedabove, the animal 902 that is presented varies according to which of thephoneme pairs are selected. If the animal 902 is a flying cow, thephoneme pair that will be presented is /Gi/ . . . /Ki/. The animal 902continues to fly around the screen until the subject places theselection hand 904 over the animal 902, and holds down a selectionbutton, such as a mouse button. After the animal 902 is presented, flowproceeds to decision block 1008.

At decision block 1008, a test is made as to whether the subject hasselected the animal 902. If not, flow proceeds to block 1010 where theanimal 902 continues to fly. The animal 902 will continue moving aboutthe scene 900 until it is selected. Flow then proceeds to block 1012.

At block 1012, the program begins presenting the selected phonemesequence. More specifically, an audio formatted file is called by theprogram that is to be played by a computer, either through speakersconnected to the computer, or though headphones worn by a subject. Inone embodiment, the file is a QuickTime audio file, configured accordingto the parameters necessary for the skill level of the user, i.e.,phoneme pair, stretching, and ISI. In addition, a starting point in thefile is chosen such that the distractor phoneme is presented a randomnumber of times, between 3 and 8 times, before the target phoneme ispresented. After the phoneme sequence begins playing, flow proceeds todecision block 1014.

At decision block 1014, a determination is made as to whether thesubject has released the animal 902. If the subject has not released theanimal 902, a parallel test is made, shown as decision block 1016.

Decision block 1016 tests whether a “hit” window has passed. Morespecifically, the program contains a lockout window of 200 ms thatbegins when the target phoneme is played. It is believed that if thesubject releases the animal 902 within 200 ms of the target phonemebeginning play, it is merely coincidental that he/she would have heardthe target phoneme. This is because no subject's reaction time is quickenough release the animal 902 so soon after hearing the target phoneme.The start of the “hit” window begins after the lockout window, i.e., 200ms after the target phoneme begins. The end of the hit window iscalculated as the start of the hit window, plus the length of onephoneme letter. So, at decision block 1016, if the hit windows has notpassed, the computer continues to test whether the subject has releasedthe animal 902. If the hit window has passed, and the subject has notreleased the animal 902, flow proceeds to block 1026.

At block 1026, a miss is recorded for that test. After recording themiss, flow proceeds back to block 1021.

At block 1021, the skill level for the selected phoneme sequence isdecreased, as will be further described below. Flow then proceeds backto block 1006 where another flying animal is presented for the samephoneme sequence.

At decision block 1014, if it is determined that the subject hasreleased the animal 902, instruction flow proceeds to decision block1018.

At decision block 1018, a determination is made as to whether the hitwindow has begun. That is, did the subject release the animal 902 duringor before the lockout period? If the hit window has not begun,instruction flow proceeds to block 1020.

Block 1020 records a false alarm and instruction flow proceeds to block1021. It should be appreciated that a false alarm is recorded, ratherthan a miss, because it suggests that the subject detected a change inthe phoneme sequence when a change has not yet occurred. If, at decisionblock 1018, the hit window has begun, flow proceeds to decision block1022.

At decision block 1022 a determination is made as to whether the hitwindow has passed. If the hit window has passed, prior to the subjectreleasing the animal 902, then flow proceeds to block 1026 where a missis recorded, as described above. However, if the hit window has notpassed flow proceeds to block 1024.

At block 1024, a hit is recorded for the subject. That is, the subjecthas correctly heard the target phoneme, and has released the animal 902in an appropriate time frame. Flow then proceeds to decision block 1028.

At decision block 1028, a determination is made as to whether thesubject has heard the target phoneme, and released the animal 902 withinthe hit window, three times in a row. If not, then flow proceeds back toblock 1006 where another animal 902 is presented. If the subject hasresponded correctly, three times in a row, flow proceeds to block 1030.

At block 1030, the skill level for the selected tone sequence isincreased by one level. In one embodiment, 18 skill levels are providedfor each phoneme sequence. As mentioned above, the skill levels begintemporal modifications of the phonemes, and by separating the presentedphonemes with an ISI of 500 ms. As the subject's ability to distinguishbetween the distractor and target phonemes improves, the temporalmodifications of the phoneme is reduced to that of normal speech, andthe ISI is reduced to 300 ms. One skilled in the art will appreciatethat the degree of phoneme temporal manipulation, from 150% to 100%, thevariation of ISI among the skill levels, and the number of skill levelsprovided, may vary depending on the LLI subject and the type of trainingthat is required. In one embodiment, after a subject successfully passesa phoneme sequence with 150% time modification, and an ISI of 500 ms,the next skill level presented holds the time modification at 150%, butreduces the ISI to 400 ms. Flow then proceeds to decision block 1032.

At decision block 1032 a determination is made as to whether the maximumlevel has been reached for the selected phoneme sequence. That is, hasthe subject progressed through all the skill levels to the point thatthey are correctly recognizing a target phoneme with a duration of 100%,and with an ISI of 0 ms? If not, then flow proceeds to block 1006 wherethe animal 902 is again presented to the subject, this time, at anincreased skill level. However, if the subject has reached the maximumlevel for a particular phoneme sequence, flow proceeds to block 1004where a phoneme tone sequence is selected. If a subject has not yetplayed the new phoneme sequence that is selected, the skill level is setto the easiest level. However, if the subject has previously heard thenew phoneme sequence, the level of play begins, either at or below thelast skill level obtained, typically 5 skill levels below what was lastobtained.

Selection of phoneme sequences and skill levels are performed by theprogram to insure that a subject is exposed to each of the phonemepairs, but spends the greater portion of his/her time with those pairsthat are the most difficult to distinguish. In addition, the number ofrecorded hits/misses/false alarms and reaction times are recorded foreach level, and for each phoneme pair, on a daily basis. The records arethen uploaded to a remote server where they are either reviewed by aremote SLP, or are tabulated and provided to a local SLP. The SLP thenhas the option of controlling the selection of phoneme sequenceselection, and/or skill level, according to the particular needs of thesubject, or of allowing automatic selection to occur in a round robinmanner.

While not shown, the program also keeps track of the number of correctresponses within a sliding window. This is visually provided to asubject by advancing the tractor 910, from the right to the left, foreach correct response. After 10 correct responses, creative animationsare played, and bonus points are awarded, to reward the subject and tohelp sustain the subject's interest in the game. Of course, the type ofanimation presented, and the number of correct responses required toobtain an animation are variables that may be set by an SLP.

Now referring to FIG. 11 a screen 1100 is shown of the second game inthe Fast ForWord program, entitled Block Commander. The Block Commandergame presents a subject with audio prompts, directing the subject toperform an action. An exemplary action might be “point to the greencircle.” The types of prompts are grouped according to difficulty,requiring a subject to perform increasingly sophisticated tasks,depending on their skill level. If the subject responds correctly he/sheis awarded a point. Otherwise, the cursor hand turns red anddemonstrates how the command should have been performed. This feedbackallows the subject to learn from the computer the more difficultmanipulations that are required. In addition, the prompts are digitallyprocessed by stretching the speech commands (in the time domain), and byemphasizing particular frequency envelopes in the speech, that containtime modulated acoustic components.

The screen 1100 contains a number score 1102 and a stop sign 1104. Thenumber score 1102 provides visual feedback to a subject regarding theirprogress in the game, and the stop sign 1104 provides a selectionmechanism for ending the game. Also shown is a cat 1106. The cat 1106provides animations for a subject during training. A grid 1120 is shown,in a 55 degree perspective, upon which are placed 3D tokens, furtherdescribed below. In the center of the grid 1120 is an ear/hand button1108. When a subject places a hand selector 1110 on top of the ear/handbutton 1108, and selects the icon (by pressing a mouse key), then atrial in the Block Commander game begins. This is shown in FIG. 12, towhich attention is now directed.

In FIG. 12, a screen shot 1200 is shown that includes the stop sign,number score, and grid, as shown above. In addition, a row of differentcolored squares 1202, and a row of different colored circles 1204 areprovided. Use of the squares 1202 and the circles 1204 will be describedbelow with reference to FIG. 13. Also shown are a number of progresstokens 1206 at the bottom of the screen 1200. The progress tokens 1206indicate the number of correct answers within a particular instance ofthe game. In one embodiment, after 5 tokens 1206 are shown, indicating 5correct responses, a reward animation and bonus points are provided tothe user.

Now referring to FIG. 13, a flow chart 1300 is shown that illustratesoperation of the Block Commander game. Execution begins at block 1302and proceeds to block 1304.

At block 1304 the game selects the first playing level that is to bepresented to a subject. To the right of block 1304 is a table 1330 thatillustrates the 5 processing levels that are used in the Block Commandergame. The levels are distinct from each other in terms of the amount ofstretching (in the time domain) that is used on speech, and the amountof emphasis that is applied to selected frequency envelopes within thespeech. Flow then proceeds to block 1306.

At block 1306, the game presents a program to a subject that trains thesubject to play the game. The training portion consists of 3 rounds. Thefirst round trains the subject to distinguish between object sizes,e.g., large and small. The second round trains the subject todistinguish between object shapes, e.g., square and circle. The thirdround trains the subject to distinguish between object colors, e.g.,blue, red, yellow, green and white. More specifically, the prompts givento a subject during training are:

Size Round 1 Touch the large circle Touch the small circle Touch thelarge square Touch the small square Shape round 2 Touch the square Touchthe circle Color round 3 Touch the blue square Touch the red squareTouch the yellow square Touch the green square Touch the white square

For a subject to pass any of the training rounds, and progress to thenext training round, two correct hits are required for each commandprompt, with no errors. If an error is made, the score is reset, andplay for that round starts over. All of the prompts for the trainingrounds are at processing level 1, 150% duration and 20 dB emphasis.After a subject has completed the training program he/she will not seeit again. Upon completion of the training program, flow proceeds todecision block 1308.

At decision block 1308 a determination is made as to whether thetraining has been completed. If not, then flow proceeds back to block1306 where training continues. If training has been completed, flowproceeds to block 1310.

At block 1310, a warm up exercise is presented to a subject. The warm upexercise is presented each time a user plays the game, at the speechprocessing level that was last completed. The warm up round includes thefollowing prompts:

Warm up Touch the green circle Touch the yellow square Touch the bluesquare Touch the white circle Touch the red circle Touch the blue circleTouch the green square Touch the yellow circle Touch the red squareTouch the white square

The ordering of the prompts is random each time the warm up is played.After presentation of each of the prompts flow proceeds to decisionblock 1312.

At decision block 1312, a determination is made as to whether the warmup round has been completed. If not, then flow proceeds back to block1310 where the warm up continues. Otherwise, flow proceeds to block1314.

At block 1314, an appropriate processing level is selected for asubject. The first time a subject plays the Block Commander game,processing level 1 is selected. However, after the subject hasprogressed beyond processing level 1, the level selected will be thelevel that the subject last played. Flow then proceeds to block 1316.

At block 1316, the first round of the game is presented to a subject. Asmentioned above, in one embodiment of the Block Commander game, sixrounds are provided. The rounds are as follows:

Round 1 Touch the green circle Touch the yellow square Touch the bluesquare Touch the white circle Touch the red circle Touch the blue circleTouch the green square Touch the yellow circle Touch the red squareTouch the yellow square Round 2 Touch the small green circle Touch thelarge red circle Touch the large white circle Touch the large red squareTouch the small yellow circle Touch the large green circle Touch thelarge green square Touch the small white circle Touch the small bluesquare Touch the large green circle Round 3 Touch the white circle andthe blue square Touch the blue square and the red circle Touch the redsquare and the green circle Touch the green square and the blue squareTouch the yellow circle and the red circle Touch the red square and thegreen square Touch the red square and the yellow circle Touch the whitesquare and the red circle Touch the green circle and the green squareTouch the blue square and the yellow circle Round 4 Touch the smallgreen circle and the large yellow square Touch the small red square andthe small yellow circle Touch the large green square and the large bluecircle Touch the large red square and the large blue square Touch thesmall red square and the small green circle Touch the small white circleand the small green circle Touch the large red square and the largewhite square Touch the large green circle and the large red circle Touchthe small blue square and the small white circle Touch the small yellowsquare and the large blue square Round 5 Put the blue circle on the redsquare Put the green square behind the white circle Touch the greencircle with the blue square Touch - with the green circle - the bluesquare Touch the green circle and the blue square Touch the green circleor the blue square Put the white square away from the yellow square Putthe yellow square in front of the red square Touch the squares, exceptthe yellow one Round 6 Put the white square beside the red circle Putthe blue circle between the yellow square and the white square Exceptfor the blue one, touch the circles Touch the red circle - No! - thegreen square Instead of the yellow square, touch the white circleTogether with the yellow circle, touch the green circle After touchingthe yellow square, touch the blue circle Put the red circle underneaththe yellow square Before touching the white circle, touch the bluesquare

Each of the prompts are presented to the user in random order, butsuccessful completion of each of the prompts in a round is requiredbefore a round is considered complete. After a first prompt is providedto a subject, flow proceeds decision block 1318.

At decision block 1318, a determination is made as to whether there havebeen 90% correct responses in a sliding group of 5 items. If not, thenflow proceeds back to block 1316 where another prompt in a round isprovided. If there have been 90% correct responses, as will beillustrated by 5 progress tokens at the bottom of the screen, then flowproceeds to block 1320.

At block 1320, the subject is shown a reward animation. In oneembodiment, the animation consists of characters morphing out of theblocks on the board. Flow then proceeds to decision block 1322.

At decision block 1322, a determination is made as to whether the roundis complete. A round is complete when a subject successfully responds toall of the prompts in the round. If the round is not complete, flowproceeds back to block 1316 where another prompt is provided to thesubject. If the round is complete, flow proceeds to decision block 1324.

At decision block 1324, a determination is made as to whether all sixrounds within the game have been completed. If not, then flow proceedsto block 1326 where the round level is incremented. Flow then proceedsback to block 1316 where prompts for the new round are presented. Ifdecision block 1324 determines that all rounds have been completed, flowproceeds back to block 1314 where an appropriate skill level isselected. In one embodiment, if a subject successfully completes all sixrounds, at skill level 1 (150% duration, 20 dB emphasis), he/she willprogress to skill level 2 (125% duration, 20 dB emphasis).

The Block Commander program begins by providing a subject with a numberof simple commands, stretched in time, with particular emphasis given tophoneme components that are difficult for an LLI subject to understand.As the subject correctly responds to the simple commands, the commandsincrease in difficulty. Once the subject masters the more difficultcommands, the amount of stretching, and the amount of emphasis isreduced, and the process is repeated. The rounds continue, over thecourse of days and weeks, until the subject is correctly responding tothe difficult commands at skill level 5, which is normal speech.

One skilled in the art will appreciate that the commands cause thesubject, not only to understand the phonemes that are presented, butalso to apply logical reasoning to the more difficult commands, and torecall the constructs of the commands. The requirement that the subjectrecall the command constructs is directed at improving the subjectsmemory, as well as to improving their ability to process acousticevents. It is believed that the games repetitive nature, that trains thesubject's neurological connections to process speech, is also helpful inimproving the subject's memory, and his/her cognitive skills inunderstanding linguistic relationships.

Now referring to FIG. 14, a screen shot 1400 is shown for the third gamein the Fast ForWord program, entitled Circus Sequence. The CircusSequence game trains a subject to distinguish between upward anddownward frequency sweeps that are common in the stop consonant portionof phonemes, by varying the duration and frequency of the sweeps, and byvarying the inter-stimulus interval (ISI) between presentation of thesweeps.

The screen 1400 contains a number score 1402, a stop sign 1404, and aprogress element 1406, all within a circus ring environment. Inaddition, the screen 1400 contains a hand selector 1408, and an ear/handbutton 1410. As in the Block Commander game, a user begins a test byselecting the ear/hand button 1410 with the hand selector 1408.

Referring to FIG. 15, a screen shot 1500 is shown that illustrates twoelements 1502, 1504 that are presented to a subject after the ear/handbutton 1410 is selected. The left element 1502 pertains to an upwardfrequency sweep, and the right element 1504 pertains to a downwardfrequency sweep. In addition, a progress element 1506 is shown elevatedabove the circus ring floor, to indicate that a subject has correctlyresponded to a number of tests. Game play will now be illustrated withreference to FIG. 16.

FIG. 16 provides a flow chart 1600 that illustrates program flow throughthe training portion of the Circus Sequence Game. Training begins atblock 1602 and proceeds to block 1604.

At block 1604, the program begins presenting a random sequence offrequency sweeps to a subject. All sweep sequences are of the form:up-up; up-down; down-up; or down-down. Thus, if the program presents thesweep sequence “up-up”, a subject is to click on the left element 1502twice. If the program presents a sweep sequence “down-up”, the subjectis to click on the right element 1504, then on the left element 1502.So, once the program provides a sweep sequence to the subject, thesubject selects the elements corresponding to the frequency modulated(FM) tone sequence. If the subject is correct, he/she is awarded points,the progress element 1506 advances upwards, and the ear/hand button 1410is presented, allowing the subject to begin another test. Duringtraining, all upward sweeps are presented starting at 1 kHz and alldownward sweeps ending at 1 kHz, with upward/downward sweeps at 16octaves per second. The duration of the sweeps are 80 ms, and the sweepsare separated by 1000 ms. Research has shown that most LLI subjects arecapable of distinguishing between frequency sweeps of this duration, andhaving an ISI of loooms. After each sweep sequence is presented, flowproceeds to decision block 1606.

At decision block 1606, a determination is made as to whether thesubject has correctly responded to 80% of the trials over a slidingscale of the last ten trials. If not, then flow proceeds back to block1604 where the sequences continue to be presented. If the subject hascorrectly responded 80% of the time, flow proceeds to block 1608.

At block 1608, random sequences are again presented, at 1 khz, having aduration of 80 ms and an ISI of 100 ms. Flow then proceeds to decisionblock 1610.

At decision block 1610, a determination is made as to whether thesubject has correctly responded to 90% of the trials over a slidingscale of the last ten trials. If not, then flow proceeds to decisionblock 1612. If the subject has correctly responded to 90% of the trialsover a sliding scale of the last ten trials, flow proceeds to block1614.

At decision block 1612, a determination is made as to whether a subjecthas correctly responded to less than 70% of the trials, over a slidingscale of the last 20 trials. If not, indicating that he/she isresponding correctly between 70-90% of the time, then flow proceeds backto block 1608 where the sweep sequences continue to be presented. If adetermination is made that the subject is correctly responding less than70% of the time over the last 20 trials, then flow proceeds back toblock 1604, where the training begins again.

At block 1614, a 3-up, 1-down rule begins. This rule allows a subject toadvance in difficulty level every time 3 correct responses are provided,while reducing the level of difficulty any time an incorrect response isgiven. Research has shown that a 3-up, 1-down rule allows a subject toobtain a correct response rate of approximately 80% near threshold,which is desired to motivate and encourage the subject to continue. Areduced accuracy rate discourages a subject, a situation that is notdesired especially if the subject is an LLI child. Once the 3-up, 1-downrule is started, flow proceeds to decision block 1616.

At decision block 1616, a determination is made as to whether a subjecthas responded correctly the last 3 tests. If so, then flow proceeds toblock 1620. If not, then flow proceeds to decision block 1618.

At decision block 1618, a determination is made as to whether a subjecthas incorrectly responded to the last test. If not, then flow proceedsback to decision block 1616 where another test is provided. However, ifthe subject has incorrectly responded to the last test, the difficultylevel is reduced one level, and flow proceeds back to decision block1616 where another test is presented. During the training level, alltests are performed at 80 ms duration, with 1000 ms ISI, which is theeasiest skill level. Therefore, if the subject incorrectly responds atthat level, no change in difficulty is made.

At block 1620, the skill level is increased. During training, the sweepsequences are presented at 1 khz, with 80 ms duration, but the ISI isreduced between the sweeps each time the level is incremented. In oneembodiment, the ISI levels start at 1000 ms, and proceed through 900ms,800 ms, 700 ms, 600 ms and 500 ms. Flow then proceeds to decision block1624.

At decision block 1624, a determination is made as to whether the ISI isat 500 ms. If not, then flow proceeds back to decision block 1616 wheresweep sequences continue to be presented. If the ISI is 500 ms, thetraining session ends and the subject is allowed to enter the real game,at block 1626.

Referring now to FIG. 17, a flow chart 1700 is provided that illustratesoperation of the Circus Sequence game, after the training session hasbeen completed. The game begins at block 1702 and proceeds to block1704.

At block 1704, an appropriate skill level is selected. The skill levelsused by Circus Sequence are shown in table 1730. For each of threefrequencies: 500 hz, 1 khz, and 2 khz, a number of skill levels areprovided. The skill levels begin by presenting frequency sweeps having aduration of 80 ms, and an ISI between the sweeps of 500 ms. As a subjectadvances, the ISI is reduced, either to 0 ms, or in one embodiment, to125 ms. It should be appreciated that the ISI increments used should beselected to slowly train a subject's ability to distinguish betweensimilar phonemes, such as /ba/ and /da/, while not frustrating thesubject by training beyond levels required to distinguish between suchphonemes.

When a subject first plays Circus Sequence, after passing training,he/she is provided with frequency sweeps beginning at 1 khz, having 80ms duration and an ISI of 500 ms. On subsequent days, the frequency thatis selected is random, and can be either 500 hz, 1 khz or 2 khz. Oncethe appropriate skill level has been selected, flow proceeds to block1706.

At block 1706, a tone sequence is presented, according to the selectedskill level. Flow then proceeds to decision block 1708.

At decision block 1708, a determination is made as to whether thesubject has correctly responded to the last 3 trials. If not, then flowproceeds to decision block 1710. If the subject has correctly respondedto the last 3 trials, flow proceeds to block 1712.

At decision block 1710, a determination is made as to whether thesubject has incorrectly responded to the last trial. If not, then flowproceeds back to block 1706 where another tone sequence is presented. Ifthe subject incorrectly responded to the last trial, flow proceeds toblock 1714.

At block 1714, the skill level is decremented. If the skill level has anISI of 500 ms, no decrease is made. However, if the skill level has anISI that is less than 500 ms, the difficulty is reduced 1 level. Forexample, if the subject incorrectly responds to a trial having an ISI of180 ms, for example, the difficulty level will be reduced, so that thenext tone sequence will have an ISI of 185 ms. Flow then proceeds backto block 1706 where another tone sequence is presented.

At block 1712, if the user has correctly responded to the last 3 trials,the skill level is incremented. For example, if a subject is at a skilllevel with a sweep duration of 80 ms and an ISI of 250 ms, the skilllevel will increase such that the ISI for the next tone sequence will be200 ms. Flow then proceeds to decision block 1716.

At decision block 1716, a determination is made as to whether the ISI isat 150 ms. If not, then flow proceeds to decision block 1720. If the ISIis at 150 ms, flow proceeds to block 1718.

At block 1718, the next lower duration is enabled. This allows theprogram to simultaneously trial a subject with multiple sweep durations,once the subject is successfully responding at an ISI level of 150 ms.For example, if a subject is correctly responding to tone sequences ofduration 80 ms, with an ISI of 150 ms, then testing continues at 80 ms.In addition, testing is begun with sweep sequences of duration 60 ms, atan ISI of 500 ms. Flow then proceeds to back to block 1706 where anothertone sequence is presented. This allows the program to present tonesequences of different duration, and different ISI, while trackingprogress for each duration/ISI combination.

At decision block 1720, a determination is made as to whether thesubject has reached a training threshold. In one embodiment, a trainingthreshold is reached when the subject has had eight skill levelreversals within six skill levels of each other. If such a threshold isreached, flow proceeds to block 1721. Otherwise, flow proceeds todecision block 1722.

At block 1721, the program moves the subject to the next frequencycategory to be tested. It is believed that once a threshold has been meton a particular day, the subject should not continue being tested at thesame frequency. Thus, the program allows a subject to progress, eitherto an ISI of 0 ms (or some other minimal ISI) or to a threshold at onefrequency, and then begin testing at an alternative frequency. Flow thenproceeds back to block 1706.

At decision block 1722, a determination is made as to whether the ISIfor a particular tone duration is 0 ms. If not, then flow proceeds backto block 1706 where another sweep sequence is presented. However, if asubject has reached a skill level of 0 ms ISI for a particular duration,flow proceeds to block 1724.

At block 1724, the program deletes the duration associated with the 0 msISI from the trial. This is because testing at that level is no longerrequired by the subject due to their proficiency. However, as mentionedabove, an alternative embodiment may select an ISI of greater than 0 msas the point where the duration is deleted from the game. Flow thenproceeds back to block 1706 where more tone sequences are presented.

While not shown, in one embodiment, a threshold level is provided thatcauses the game to begin testing a subject at an alternate frequency.For example, if the subject is testing at 500 hz, and a threshold isreached, the program will begin testing the subject at 2 khz. Thethreshold is reached when a subject has 8 skill level reversals within 6levels of each other. When this occurs, the program ceases testing atthe frequency for which the threshold was reached, and begins testing atan alternative frequency.

Also, when a subject begins each day of testing, a frequency differentthan that tested the previous day is begun. Moreover, a skill level thatis 5 less than completed the previous day is chosen, presuming thesubject completed at least 20 trials for that frequency.

As mentioned above, each correct response causes the progress element1506 to advance upward. After ten correct responses, a reward animationis provided to entertain the subject. When the animation ends, thesubject is prompted with the ear/hand button 1410 to begin anothertrial.

Now referring to FIG. 18, a screen shot 1800 of the fourth game in FastForward, Phonic Match, is provided. The screen 1800 includes a set ofpictures 1802, a progress creature 1804, a stop sign 1806, and a numberscore 1808. The progress creature 1804, stop sign 1806 and number score1808 function similarly to those described in previous games.

The set of pictures 1802 are arranged into a 2×2 grid. When a subjectselects any of the pictures, a word or phoneme is played. On any grid,there are two pictures that play the same word. Thus, for a 2×2 grid,there are two words that will be presented. The test for the subject isto distinguish between similar words, to recall which picture isassociated with which word, and to sequentially select two pictures thatpresent the same word. Similar words are presented together, with thewords processed according to the processing levels shown in table 1902of FIG. 19.

Initially, subjects are presented words at processing level 1, with aduration of 150%, and having 20 dB emphasis of selected frequencyenvelopes within the words. In addition, different skill levels, asshown in table 1904, are provided that increase the grid size for aparticular trial, and set the maximum number of clicks, or selections,that a subject can attempt before losing the trial. Operation of thegame is illustrated in FIG. 20. However, before providing a detaileddescription of game operation, the words used in the game are shown.

Word Group 1 big, bit, dig, dip, kick, kid, kit, pick, pig, pit, tick,tip Word Group 2 buck, bud, but, cup, cut duck, dug, pub, pup, tub,tuck, tug Word Group 3 back, bag, bat, cab, cap, cat, gap, pack, pat,tack, tag, tap Word Group 4 ba, cha, da, ga, ka, la, pa, ra, sa, sha,ta, za

Referring now to FIG. 20, the Phonic Match game begins at block 2002,and proceeds to block 2004.

At block 2004, a 2×2 grid is presented. The words associated with the2×2 grid are selected from one of the four Word Groups shown above. Theselection of the Word Group is random, except that tracking ofpreviously played Word Groups is done to insure that all Word Groups areequally represented, and that a subject is not provided the same WordGroup as played on an immediately preceding day. The words within a WordGroup are typically selected according to their acoustic similarity.

The subject is required to sequentially select two pictures that havethe same word associated with them. When a subject sequentially selectstwo pictures associated with the same word, the pictures are removedfrom the gird being played. After a subject completes a 2×2 grid,whether correctly or incorrectly, flow proceeds to decision block 2006.

At decision block 2006, a determination is made as to whether thesubject has successfully passed three 2×2 grids. Referring to table 1904of FIG. 19, ten skill levels are shown. When a 2×2 grid is firstpresented, the skill level entered is level 8. Skill level 8 defines a2×2 grid, with a maximum number of allowed clicks as 8. If a subjectselects pictures on a 2×2 grid more than 8 times, the grid is notconsidered passed, and game flow proceeds back to block 2004 whereanother grid is presented. If not, then flow proceeds back to block 2004where another 2×2 grid is presented with words from the same Word Group.If the subject has successfully passed three 2×2 grids, thus progressingfrom level 8 through level 10, flow proceeds to block 2008.

At block 2008, a new grid is presented for a particular Word Group, orstimulus set. Initially, a 3×3 grid is provided, at skill level 2. Themaximum number of clicks allowed for a subject to pass a 3×3 grid is 20.Within a 3×3 grid, 1 of the pictures is a wildcard, since there are anodd number of pictures. Selection of the wildcard simply removes thepicture from the grid, and does not count against the subject as aselection, or click. After a 3×3 grid is presented to a subject, flowproceeds to decision block 2010.

At decision block 2010, a determination is made as to whether thesubject passed the level. That is, did the subject properly distinguishbetween word pairs, and sequentially select picture pairs associatedwith words in 20 or less clicks. If so, then flow proceeds to block2012. If not, then flow proceeds to block 2014.

At block 2012, the skill level is incremented. For example, if a subjectwas at level 2, he/she will increment to level 3. Note: levels 2-3present a 3×3 grid with a maximum number of clicks of 20, while levels4-7 present a 4×4 grid with a maximum number of clicks of 60. Once theskill level is incremented, flow proceeds to block 2020.

At block 2020, a grid according to the new skill level is presented. Thegrid is associated with the same Word Group that was previously used,but possibly with different words from the group. Flow then proceeds todecision block 2022.

At decision block 2022, a determination is made as to whether thesubject has passed the level. That is, did the subject correctlyassociate the word pairs in less than or equal to the number of allowedclicks. If not, flow proceeds to block 2014. If the subject passed thelevel, flow proceeds to decision block 2024.

At decision block 2024, a determination is made as to whether thesubject has reached skill level 7. Level 7 is termed the “decision”level. If the skill level that has just been passed is not level 7, thenflow proceeds back to block 2012 where the skill level is incremented.However, if the skill level passed is level 7, flow proceeds to decisionblock 2026.

At decision block 2026, a determination is made as to whether all fourstimulus sets, or Word Groups have been passed. If not, then flowproceeds to block 2018. However, if a subject has correctly passed skilllevel 7, for all four Word Groups, flow proceeds to block 2028.

At block 2028, the next processing level is selected. Referring to table1902 of FIG. 19, a subject begins at processing level 1 (duration 150%,emphasis 20 dB). Once all four Word Groups have been passed at skilllevel 7, the amount of audio processing to the words is reduced. First,the duration of the words is reduced, from 150%, to 125%, to 100%, andthen the amount of emphasis applied to selected frequency components isreduced, from 20 dB, to 10 dB, to 0 dB. Once a subject has reachedprocessing level 5, he/she is presented with normal speech. After thenext processing level is selected, flow proceeds to decision block 2030.

At decision block 2030, a determination is made as to whether allprocessing levels have been completed. That is, has the subject reachedprocessing level 5. If not, flow proceeds back to block 2004 where thegame begins anew, with a 2×2 grid, but at the new processing level.However, if the subject has reached processing level 5, flow proceeds toblock 2032.

At block 2032, a 5×5 grid is provided, with a maximum number ofallowable clicks as 90. From this point forward, the game continuesplaying indefinitely, but the decision round, level 7, switches from a4×4 grid to a 5×5 grid.

Referring back to decision block 2022, if a subject does not pass aparticular level, flow proceeds to block 2014.

At block 2014, the skill level is decremented. Flow then proceeds todecision block 2016.

At decision block 2016, a determination is made as to whether the newskill level is less than level 1. Level 1 is considered a “slip” levelindicating that if a user failed at this level, a new Word Group shouldbe provided. If the skill level is not less than 1, flow proceeds backto block 2020 where a new grid is presented, according to the presentlevel. If the new level is less than 1, that is, if the subject failedto pass a grid, at skill level 1, flow proceeds to block 2018.

At block 2018, the program discontinues presenting words from thepresent Word Group, and changes the Word Group used for the grids. Flowthen proceeds back to block 2008 where a 3×3 grid is presented, at skilllevel 2, using words from the new Word Group.

The flow chart 2000 demonstrates that a subject is required to proceedfrom level 2 through level 7 for each of the four Word Groups, at aparticular processing level, before he/she is allowed to advance to thenext processing level. The progress creature descends with each click.If the creature reaches the bottom, then the grid is not passed. If allpicture pairs are matched prior to the creature reaching the bottom,extra points are awarded, a reward animation is presented and the gridis considered passed. When a subject has correctly selected apredetermined number of picture pairs, the progress animal 1804 reachesthe top, and the subject is rewarded by an animation.

Referring now to FIG. 21, a screen shot 2100 is shown illustrating thefifth game in the Fast Forward program, entitled Phonic Words. PhonicWords presents a subject with a sentence prompt that requires thesubject to distinguish between two similar words, to accurately selectone of two pictures 2108, 2110, using a selection hand 2112. The tablebelow provides a list of the word pairs used. The first word in the pairis always the correct answer, but its representational image couldappear on the left or right of the screen 2100.

base-face, face-base, vase-base, base-vase, face-vase, vase-face,bee-me, me-bee, knee-bee, bee-knee, knee-me, me-knee, breathe-breeze,breeze-breathe, day-they, they-day, lawn-yawn, yawn-lawn, ache-lake,lake-ache, ache-rake, rake-ache, ache-wake, wake-ache, lake-rake,rake-lake, lake-wake, wake-lake, rake-wake, wake-rake, sink-think,think-sink, chip-dip, dip-chip, sip-zip, zip-sip, chip-sip, sip-chip,chip-zip, zip-chip, dip-sip, sip-dip, dip-zip, zip-dip, pack-shack,shack-pack, tack-shack, shack-tack, pack-tack, tack-pack, tack-tag,tag-tack, rung-young, young-rung, rung-run, run-rung, young-run,run-young, pat-path, path-pat, bear-bell, bell-bear, thumb-tongue,tongue-thumb, comb-cone, cone-comb, mouse-mouth, mouth-mouse,cash-catch, catch-cash, fan-fang, fang-fan, sauce-saws, saws-sauce,bass-bath, bath-bass, cheese-chief, chief-cheese, foam-phone,phone-foam, fuzz-fudge, fudge-fuzz, safe-shave, shave-safe, long-lawn,lawn-long, piece-peas, peas-piece, piece-peach, peach-piece, peas-peach,peach-peas, wash-watch, watch-wash.

As before, the screen 2100 contains an ear/hand button 2102 forbeginning a trial, a stop sign 2104 for ending the game, and a numberscore 2106. Within the number score 2106 are five acorns, indicating theprocessing level currently being tested. Also shown are progresscreatures 2114 indicating a number of correct responses. As a subjectcorrectly responds to the game, a new progress creature 2114 is added.When the number of progress creatures 2114 reaches ten, a rewardanimation is provided and bonus points are awarded.

Referring to FIG. 22, a screen shot 2200 is shown where the word pairpeach-peas is being tested. After a subject listens to a promptcontaining the target word, he/she selects one of the two pictures. Thesubject, whether correct or incorrect, will then be shown the correctselection, in this case peach, by having the mask removed from thepicture frame 2202.

Referring now to FIG. 23, operation of the Phonic Words game isillustrated by flowchart 2300. Please note that five processing levels,similar to those used above in Phonic Match and Block Commander, areshown in table 2340. The game begins at block 2302 and proceeds totraining block 2304.

At training block 2304 the subject is prompted to “press the earbutton”. The prompting is processed at level 1 (duration 150%, emphasis20 dB). Flow then proceeds to decision block 2306.

At decision block 2306, a determination is made as to whether theear/hand button 2102 has been pressed. If not, then flow proceeds backto block 2304 where the prompting is repeated. If the ear/hand button2102 has been pressed, flow proceeds to block 2308.

At block 2308, praise is played for the subject. Flow then proceeds toblock 2310.

At block 2310, a single image appears in one of the two frames 2108,2110, and a sound file pertaining to the image is played for thesubject. Flow then proceeds to decision block 2312.

At decision block 2312, a determination is made as to whether thesubject has selected the appropriate image. The image continues to bedisplayed until the subject selects the image. Flow then proceeds todecision block 2314.

At decision block 2314, a determination is made as to whether thesubject has correctly selected the single image, three times. If not,then flow proceeds back to block 2310 where another image is presented,with its associated word. If the subject correctly selects an image/wordcombination three times, flow proceeds to block 2316.

At block 2316, a pair of images are presented, along with a commandprompt containing a word associated with one of the images. The otherimage presented is termed the distractor image. The user must click onthe correct image 4 out of 5 times in a sliding scale to start the game.After the double image is presented, flow proceeds to decision block2318.

At decision block 2318, a determination is made as to whether thesubject has correctly selected an image, from the image pair, in 4 outof 5 cases, on a sliding scale. If not, then flow proceeds back to block2316 where another image pair is presented. Otherwise, flow proceeds toblock 2320 where the subject enters the game. Flow then proceeds toblock 2322.

At block 2322, a subject is presented a sequence of image pairs, withassociated words selected from a particular processing set. Theprocessing sets are chosen by grouping words having similar phonemecharacteristics. Once all of the words have been presented within aprocessing set, flow proceeds to decision block 2324.

At decision block 2324, a determination is made as to whether thesubject has correctly understood a word, and properly selected itsassociated picture from the picture pair with 90% or greater accuracy.If not, flow proceeds back to block 2322 where random selection ofimage/word pairs continue, until a 90% success rate is achieved. Flowthen proceeds to block 2326.

At block 2326, a new processing set is selected. Flow then proceeds todecision block 2328.

At decision block 2328, a determination is made as to whether all of theprocessing sets have been completed. If not, then flow proceeds back toblock 2322 where random selection of image/word pairs are presented fromthe current processing set. However, if all of the processing sets havebeen completed, flow proceeds to block 2330.

At block 2330, the processing level is incremented. Initially, theprocessing level is level 1. After a subject has completed all of theprocessing sets, with a 90% or greater accuracy for each of the sets,the processing level is increased to level 2. As described above, theduration of the words is decreased first, from 150%, to 125% to 100%,and then the emphasis of selected frequency envelopes is reduced, from20 dB, to 10 dB, to 0 dB, until normal speech (level 5) is obtained.After the processing level is incremented, flow proceeds to decisionblock 2332.

At decision block 2332, a determination is made as to whether a subjecthas completed all of the sets at processing level 5. If not, then flowproceeds back to block 2322 where random selection of image/word pairswithin a set are presented at the new processing level. However, if thesubject has completed all of the processing sets at level 5, flowproceeds to block 2334.

At block 2334, Phonic Words continues to drill the subject randomlyselecting image/word pairs within a processing set, at level 5.

Now referring to FIG. 24, a screen shot 2400 is provided for the sixthgame in the Fast ForWord program, entitled Phoneme Identification.Phoneme Identification processes a number of phoneme pairs byselectively manipulating parameters such as consonant duration,consonant emphasis, and inter-stimulus interval. More specifically, fivephoneme pairs are tested, each pair containing a target sound and adistractor. These include: 1) aba-ada; 2) ba-da; 3) be-de; 4) bi-di; and5) va-fa.

For each phoneme pair, 26 different skill levels are provided, eachlevel differing from the other in the degree of processing applied(duration and emphasis), and in the separation (ISI) of the distractorand target phoneme. Skill level 1 processes the phoneme pair bystretching the consonant portion 150% while leaving the vowel portionuntouched, emphasizing selected frequency envelopes in the consonantportion 20 dB, and separating the distractor and target phonemes by 500ms, for example. Skill level 26 provides a phoneme pair withoutstretching or emphasis, and with an ISI of 0 ms. Skill levels 2-25progress towards normal speech by applying less and less consonantprocessing, with less and less separation between the distractor andtarget phonemes.

The screen 2400 contains an ear/hand button 2402 to allow a subject tobegin a trial, a number score 2404 for tracking correct responses, astop sign 2406 for exiting the game, a hand selector 2408, and progresselements 2410 for graphically illustrating progress to a subject. Whenthe game is initially selected, five different animals are shown on thescreen, each pertaining to a phoneme pair to be tested. A subject mayselect any one of the five animals to begin the game. After a subjecthas played the game with one of the five animals, the choice is reducedto four animals, and so on.

Referring to FIG. 25, a screen shot 2500 is shown with two polar bears2502, 2504. In one embodiment, the polar bears 2502, 2504 are associatedwith the phoneme pair ba-da. There are five background scenes, eachassociated with an animal/phoneme pair, each having their ownanimations, etc. When a subject presses the ear/hand button 2402, thegame plays a target phoneme, either ba or da. The phoneme pair is thenpresented by the polar bears 2502, 2504 with one bear speaking thedistractor and the other bear speaking the target sound. A subject isrequired to distinguish between the distractor and target phonemes, andto select with the hand selector 2508, the polar bear that spoke thetarget phoneme. Details of how the game Phoneme Identification is playedwill now be provided with reference to FIGS. 26 and 27.

Referring to FIG. 26, a flow chart 2600 is shown that illustrates thetraining module of the Phoneme Identification game. Training begins atblock 2602 and proceeds to block 2604.

At block 2604, the game presents the screen shot 2400, and prompts asubject to “press the ear button”. Flow then proceeds to decision block2606.

At decision block 2606, a determination is made as to whether thesubject has pressed the ear/hand button 2402. If not, then flow proceedsback to block 2604 where the prompt is repeated, after a predeterminedinterval. If the subject has pressed the ear/hand button 2402, flowproceeds to block 2608.

At block 2608, the ear/hand button 2402 is presented, but this timewithout an audio prompt. Flow then proceeds to decision block 2610.

At decision block 2610, a determination is made as to whether thesubject has pressed the ear/hand button 2402. If not, then flow proceedsback to block 2608. The subject remains in this loop until the ear/handbutton 2402 is pressed. Once the ear/hand button 2402 is pressed, flowproceeds to block 2612.

At block 2612, a target phoneme, pertaining to a selected animal pair,is played for a subject. The target phoneme is processed at level 1,150% duration, with 20 dB emphasis, as shown by the table 2640. Flowthen proceeds to block 2614.

At block 2614, a single animal is presented that speaks the targetphoneme. Flow then proceeds to decision block 2616.

At decision block 2616, a determination is made as to whether the animalthat spoke the target phoneme has been selected. If not, flow proceedsback to block 2614 where the animal again speaks the target phoneme,after a predetermined interval. However, if the subject has selected theanimal, flow proceeds to decision block 2618.

At decision block 2618, a determination is made as to whether thesubject has correctly pressed the animal in ten trials. If not, thenflow proceeds back to block 2612 where another trial is begun. However,once the subject has correctly responded in ten trials, flow proceeds toblock 2620.

At block 2620, a target phoneme is again presented, at level 1processing. Flow then proceeds to block 2622.

At block 2622, two animals are now presented, one speaking the targetphoneme, the other speaking the distractor phoneme. The order ofspeaking the target and distractor phonemes is random, with the animalon the left speaking first, and the animal on the right speaking last.However, in this training level, the animal that speaks the targetphoneme is visually highlighted for the subject. Both the target anddistractor phonemes are processed at level 1, and are separated in timeby 500 ms. Flow then proceeds to decision block 2624.

At decision block 2624, a determination is made as to whether thesubject has correctly selected the animal speaking the target phoneme in8 out of 10 trials, on a sliding scale. If not, then flow proceeds backto block 2620 where another trial is begun. If the subject has correctlyresponded in 8 out of 10 trials, flow proceeds to block 2626.

At block 2626, a target phoneme is presented to a subject, processed atlevel 1. Flow then proceeds to block 2628.

At block 2628, two animals are shown presenting a target phoneme and adistractor phoneme, both processed at level 1, with an ISI of 500 ms.The order of target/distractor phonemes is random. For this trial,however, the animal speaking the target phoneme is not visuallyhighlighted for the subject. Flow then proceeds to decision block 2630.

At decision block 2630, a determination is made as to whether thesubject has correctly responded to 8 out of 10 trials, on a slidingscale. If so, then the subject has successfully completed the trainingand flow proceeds to block 2634, allowing the subject to advance to thegame. However, if the subject has not been successful in 8 out of 10trials, then flow proceeds to decision block 2632.

At decision block 2632, a determination is made as to whether thesubject has responded correctly less than 70% of the time in at least 10trials. If not, then flow proceeds back to block 2626 where anothertrial is presented. If the subject has less than a 70% success rate,over at least 10 trials, then flow proceeds back to block 2614 wheretrials begin again, but where visual highlighting of the animal speakingthe target phoneme is provided for the subject.

Referring now to FIG. 27, a flow chart 2700 is provided that illustratesplay of the Phoneme Identification game. Play begins at block 2702 andproceeds to decision block 2704.

At decision block 2704, a determination is made as to whether theear/hand button 2402 has been pressed. If not, then flow proceeds backto decision block 2704 until the subject chooses to hear the targetphoneme. If the ear/hand button 2402 has been pressed, flow proceeds toblock 2706.

At block 2706 a target phoneme is presented at an appropriate processinglevel. If this is the first time a subject has played the game, then theprocessing level for the phonemes is level 1, and the ISI between thetarget and distractor phonemes is 500 ms. Otherwise, the skill levelpertains to the historical success of the subject, with the particularphoneme pair, as will be further described below. Flow then proceeds toblock 2708.

At block 2708, two animals are shown, corresponding to the phoneme pairbeing tested, speaking the processed target and distractor phonemes, inrandom order. Flow then proceeds to decision block 2710.

At decision block 2710, a determination is made as to whether thesubject has correctly selected the animal speaking the target phoneme.If not, then flow proceeds to block 2720. If the subject has correctlyresponded to the trial, flow proceeds to decision block 2712.

At block 2720, the skill level for play is decremented. For example, ifthe processing level is at level 1, having consonant duration of 150%,and emphasis of 20 db, but the ISI between the target and distractorphonemes is at 100 ms, the game will drop back to a skill level wherethe ISI is at 110 ms. However, if the skill level of play is already atlevel 1, then no change in processing is made.

At decision block 2712, a determination is made as to whether thesubject has correctly responded in the last 3 consecutive trials. Ifnot, then flow proceeds back to decision block 2704, awaiting anothertrial to begin. However, if the subject has correctly responded to thelast 3 trials, flow proceeds to block 2714. It should be appreciatedthat the procedure illustrated in blocks 2710-2712 is the 3-up, 1-downrule, previously described in the Circus Sequence game above.

At block 2714, the skill level of the game is incremented. For example,if a subject has correctly responded to 3 consecutive trials, and is ata processing level of 100% duration, 20 dB emphasis, and an ISI of 0 ms,the next level of play will be at 100% duration, 10 dB emphasis, and anISI of 500 ms. Flow then proceeds to decision block 2716.

At decision block 2716, a determination is made as to whether thehighest skill level has been reached. If the subject has correctlyresponded to the last 3 trials, with no processing of the phonemes, andwith minimal ISI between the target and distractor, then flow proceedsto block 2718. Otherwise flow proceeds to decision block 2722.

At decision block 2722, a determination is made as to whether thesubject has reached a threshold. In one embodiment, a threshold isreached if the subject has had 8 skill level reversals within 6 skilllevels of each other. If the subject has not reached a threshold, flowproceeds back to block 2704 where another trial is begun. If the subjecthas reached a threshold, flow proceeds to block 2718.

At block 2718, a new stimulus category is selected. That is, a newphoneme pair is selected for testing. Thus, if the subject has beentested with the phoneme pair ba-da, and has either mastered the pair byreaching the highest skill level, or has reached a threshold, then analternate phoneme pair is selected, say aba-ada. Flow then proceeds backto block 2704 where a trial awaits using the new phoneme pair. In oneembodiment, the skill level used for the new phoneme pair is selected tobe 5 less than previously achieved for that pair. Or, if the subject hasnot yet been tested on the new phoneme pair, the skill level is setto 1. Testing continues indefinitely, or for the time allotted forPhoneme Identification on the subject's daily training schedule.

Referring now to FIG. 28, a screen shot 2800 is shown for the seventhgame in the Fast Forward program, Language Comprehension Builder. Thescreen shot 2800 contains an ear/hand button 2802 for beginning a trial,a stop sign 2804 for exiting the game, a number score 2806 correspondingto the number of correct responses, and level icons 2808 for indicatingthe processing level that is currently being tested. In addition, fourwindows 2810 are shown for containing one to four stimulus images,according to the particular trial being presented. If less than fourstimulus images are required for a trial, they are placed randomlywithin the four windows 2810. At the bottom of the screen 2800 aresmaller progress windows 2812 for holding progress elements. Theprogress elements provide a visual indicator to a subject of his/herprogress. As in previously discussed games, when all of the progresselements are obtained, usually ten correct responses, a reward animationis presented to the subject. In one embodiment of this game, the rewardanimation builds a space ship out of the progress elements.

The stimulus that is provided to the subject is in the form of commandsentences. The sentences are divided into 7 comprehension levels, witheach level having between 4 to 10 groups of sentences. Each group has 5sentences. For each stimulus sentence, a corresponding image isprovided, with 1-3 distractor images. The subject is to listen to thestimulus sentence and select the corresponding image.

Each of the stimulus sentences may be processed by stretching words, orselected phonemes, in time, and by emphasizing particular frequencyenvelopes, as shown by table 3040 in FIG. 30. Stretching and emphasis ofselected words/phonemes is similar to that described above in othergames. The stimulus sentences presented to a subject are provided inAppendix A.

Referring now to FIG. 29, a flow chart 2900 is provided to illustratethe training tutorial aspect of the game. Training begins at block 2902and proceeds to block 2904.

At block 2904, the subject is prompted to “press the yellow button”.That is, the ear/hand button 2802. Flow then proceeds to decision block2906.

At decision block 2906, a determination is made as to whether thesubject has selected the ear/hand button 2802. If not, flow proceedsback to block 2904 where the subject is again prompted, after apredetermined interval. If the subject has pressed the button, flowproceeds to block 2908.

At block 2908, the ear/hand button 2802 is presented, without audioprompting. Flow then proceeds to decision block 2910.

At decision block 2910, a determination is made as to whether thesubject has pressed the button 2802. If not, then the subject stays inthis loop until the button 2802 is pressed. Once pressed, flow proceedsto block 2912.

At block 2912, a subject is presented with a single image andcorresponding audio stimulus. In one embodiment, the stimulus isprocessed at level 1, with 150% duration and 20 dB selective emphasis.Flow then proceeds to decision block 2914.

At decision block 2914, a determination is made as to whether thesubject has selected the image corresponding to the presented stimulus.If not, then flow proceeds back to block 2912 where the subject is againprompted with the stimulus, after a predetermined interval. However, ifthe subject selected the image, flow proceeds to decision block 2916.

At decision block 2916, a determination is made as to whether thesubject has correctly selected an image, 3 times. If not, then flowproceeds back to block 2912 where another image/stimulus combination ispresented. However, if the subject has correctly selected an image, 3times, flow proceeds to block 2918.

At block 2918, an image/stimulus combination is presented, along with adistractor image. Flow then proceeds to decision block 2920.

At decision block 2920, a determination is made as to whether thesubject selected the appropriate image. if not, then flow proceeds backto block 2918. However, if the subject selected the correct image, flowproceeds to decision block 2922.

At decision block 2922, a determination is made as to whether thesubject has correctly responded to 4 out of 5 trials, on a slidingscale. If not, then flow proceeds back to block 2918. If the subject hascorrectly responded 4 out of the last 5 trials, flow proceeds to block2924 allowing the subject to start the game.

Now referring to FIG. 30, a flowchart 3000 is shown illustratingoperation of the Language Comprehension Builder game. The game begins atblock 3002 and proceeds to block 3004.

At block 3004 an image and stimulus combination is presented to thesubject. In one embodiment, the game begins by selecting a group fromLevel 2, and then by randomly selecting one of the trials from theselected group. The processing of the sentence is performed at 150%duration with 20 dB selected emphasis. Flow then proceeds to decisionblock 3006.

At decision block 3006, a determination is made as to whether thesubject correctly selected the image associated with the stimulussentence. If not, the subject is shown the correct response, and flowproceeds back to block 3004 where another stimulus/image combinationfrom the same group is presented. If the subject selects the correctimage, flow proceeds to decision block 3008.

At decision block 3008, a determination is made as to whether allsentences within a stimulus set have been successfully completed. Asmentioned above, the program begins in Level 2, by selecting aparticular stimulus set for presentation. The program stays within theselected stimulus set until all stimulus sentences have been respondedto correctly. The program then selects another stimulus set from withinLevel 2. If the subject has not correctly completed all sentences withina stimulus set, flow proceeds back to block 3004 where another sentenceis presented. If the subject has completed all stimulus within a set,flow proceeds to decision block 3010.

At decision block 3010, a determination is made as to whether all setswithin a particular comprehension level have been completed. If not,then a new set is selected, and flow proceeds back to block 3004.However, if all sets within a comprehension level have been completed,flow proceeds to block 3012.

At block 3012, the comprehension level is incremented. In oneembodiment, a subject proceeds through comprehension levels 2-6, inorder, with levels 7 and 8 interspersed within levels 3-6. Flow thenproceeds to decision block 3014.

At decision block 3014, a determination is made as to whether allcomprehension levels have been completed. If not, then flow proceedsback to block 3004 where the subject is presented with an image/stimuluscombination from a stimulus set within the new comprehension level.However, if the subject has progressed through all stimulus sets for allcomprehension levels, flow proceeds to block 3016.

At block 3016, the processing level applied to the stimulus sets isincreased. The processing levels are shown in table 3040. For example,if a subject has just completed processing level 2, having a duration of125%, and 20 dB emphasis, the processing level is incremented to level3. This will present all stimulus at 100% duration, and 20 dB emphasis.In addition, it will reset the comprehension level to level 2, and willrestart the stimulus set selection. Flow then proceeds to decision block3018.

At decision block 3018, a determination is made as to whether allprocessing levels have been completed. If not, then flow proceeds backto block 3004 where a stimulus set from level 2 is presented to thesubject, at the new processing level. However, if all the processinglevels have been completed, the subject remains at processing level 5(normal speech). Flow then proceeds to block 3020.

At block 3020, the comprehension levels are reset, so that the subjectis presented again with stimulus from level 2. However, no alteration inthe stimulus is performed. The subject will remain at processing level5.

Study has shown that several weeks are required for a subject to advancethrough all of the comprehension levels, and all of the processinglevels. Therefore, when a subject begins each day, he/she is startedwithin the comprehension level, and stimulus set that was last played.And, the stimulus set will be presented at the processing level lastplayed.

In Language Comprehension Builder, as in all of the other games,detailed records are kept regarding each trial, indicating the number ofcorrect responses and incorrect responses, for each processing level,skill level and stimulus set. These records are uploaded to a centralserver at the end of each day, so that a subject's results may betabulated and analyzed by an SLP, either working directly with asubject, or remotely. Based on analysis by the SLP, modification totraining parameters within Fast ForWord may be made, and downloaded tothe subject. This allows a subject to begin each day with a sensorytraining program that is individually tailored to his/her skill level.

The above discussion provides a detailed understanding of the operationof the present invention as embodied in the game modules within theprogram entitled Fast ForWord. Each of the game modules presentdifferent problems to a subject, using modified phonemes, frequencysweeps or speech commands that are stretched, emphasized or separated intime, according to the subject's ability, and according to predefinedprocessing parameters within the program. Although alternative acousticprocessing methodologies may be used, discussion will now be directed atalgorithms developed specifically for use by the above described games.

In one embodiment, a two-stage speech modification procedure was used.The first stage involved time-scale modification of speech signalswithout altering its spectral content. The time scale modification iscalled the “phase vocoder”, and will be further described below. Thesecond speech modification stage that was developed uses an algorithmthat differentially amplifies and disambiguates faster phonetic elementsin speech. “Fast elements” in speech are defined as those that occur inthe 3-30 Hz range within an envelope of narrow-band speech channels of arate changed speech signal. An emphasis algorithm for these fastelements was implemented using two methods: a filter-bank summationmethod and an overlap-add method based on a short-time Fouriertransform. Both of these emphasis algorithms will be further describedbelow.

Time-scale Modification

Referring to FIG. 31, a flow chart 3100 is provided that illustratestime-scale modification of speech signals according to the presentinvention. Modification begins at block 3102 and proceeds to block 3104.

At block 3104, segmented digital speech input is provided to aprocessor. The segmented speech is assumed to be broadband and composedof a set of narrow-band signals obtained by passing the speech segmentthrough a filter-bank of band-pass filters. The speech signals may bewritten as follows:${f(t)} \cong {\sum\limits_{n = 1}^{N}{f_{n}(t)}}$

where f_(n)(t) = ∫_(−∞)^(t)f(t)h(t − τ)cos [ω_(n)(t − τ)]τ

This is the convolution integral of the signal f(t) and h(t), aprototypical low-pass filter modulated by cos[ω_(n)(t)] where ω_(n) isthe center frequency of the filters in the filter-bank, an operationcommonly referred to as heterodyning. Flow then proceeds to block 3106.

At block 3106, the above integral is windowed, and a short-term Fouriertransform of the input signal is evaluated at the radian frequency ω_(n)using an FFT algorithm. The complex value of this transform is denoted:

ƒ_(n)(t)=|F(ω_(n) ,t)|cos[ω_(n) t+φ _(n)(ω_(n) ,t)]

where φ_(n)(ω_(n), t) is the phase modulation of the carriercos[ω_(n)(t)]. Flow then proceeds to block 3108.

At block 3108 the amplitude and phase of the STFT is computed. It isknown that the phase function is not a well behaved function, howeverits derivative, the instanteneous frequency, is bounded and is bandlimited. Therefore, a practical approximation for f_(n)(t) is:f_(n)(t) ≅ F(ω_(n), t)cos [ω_(n)t + ∫₀^(t)ϕ_(n)^(*)(ω_(n), t)]

where φ* is the instantaneous frequency. Flow then proceeds to block3110.

At block 3110 φ* can be computed from the unwrapped-phase of theshort-term Fourier transform. A time-scaled signal can then besynthesized as follows by interpolating the short-term Fourier transformmagnitude and the unwrapped phase to the new-time scale as shown below.$\left. {{f\left( {\beta \quad t} \right)} \cong {\sum\limits_{n = 0}^{\beta \quad N}{{{F\left( {\omega_{n},{\beta \quad t}} \right)}}\cos\left( \beta \left( {{\omega_{n}t} + {\int_{0}^{t}{\phi_{n}^{*}\left( {\omega_{n},t} \right)}}} \right) \right.}}} \right)$

where β is the scaling factor which is greater than one for time-scaleexpansion. An efficient method to compute the above equation makes useof cyclic rotation and the FFT algorithm along with an overlap-addprocedure to compute the short-time discrete Fourier transform.Appropriate choice of the analysis filters h(t) and interpolatingfilters (for interpolation of the short-term Fourier transform to thenew time-scale) are important to the algorithm. In one embodiment,linear interpolation based on the magnitude and phase of the short-timeFourier transform was used. The analysis filter h(t) was chosen to be aKaiser window multiplied by an ideal impulse response as shown:${h(n)} = {\frac{N}{\pi \quad n}{\sin \left( \frac{\pi \quad n}{N} \right)}{{kaiser}\left( {n,6.8} \right)}}$

where${{{kaiser}\left( {n,\alpha} \right)} = \left\{ \frac{I_{o}\left\lbrack {\alpha \sqrt{\left( {1 - \left\lbrack {{\left( {n - {N/2}} \right)/N}/2} \right\rbrack^{2}} \right.}} \right\rbrack}{I_{o}(\alpha)} \right\}},\quad {0 \leq n \leq N}$

where I_(o)(α) is the zeroth-order modified Bessel function of the firstkind and N is the length of the analysis window over which the FFT iscomputed. Flow then proceeds to block 3112.

At block 3112, a short-term inverse FFT is computed to produce digitalspeech output. This output is then provided at block 3114.

Filter-bank Emphasis Algorithm

Now referring to FIG. 32, a flow chart 3200 is shown that illustratesimplementation of an emphasis algorithm according to the presentinvention. The algorithm begins at block 3202 and proceeds to block3204.

At block 3204, it is assumed that the speech signal can be synthesizedthrough a bank of band-pass filters, as described above. This time,however, no heterodyning of a prototypical low-pass filter is used.Instead, a set of up to 20 second-order Butterworth filters with centerfrequencies logarithmically spaced between 100 and the nyquist frequencyare used. The output of each band-pass filter resulted in a narrow-bandchannel signal f_(n)(t). Flow then proceeds to block 3206.

At block 3206, we computed the analytical signal as follows:

a _(n)(n)=ƒ_(n)(n)+iH(ƒ_(n)(n))

where H(n) is the Hilbert transform of a signal defined as:${H(n)} = {{{f_{n}(n)}*\left( \frac{1}{\pi \quad t} \right)} = {\int{{f_{n}(\tau)}\quad \frac{1}{\pi \left( {n - \tau} \right)}{\tau}}}}$

The Hilbert transform was computed using the FFT algorithm. It is knownthat the absolute value of the analytical signal is the envelope of anarrow-band signal. Thus, an envelope e_(n)(n) is obtained by thefollowing operation:

e _(n)(n)=|a _(n)(n)|

The envelope within each narrow-band channel is then band-pass filteredusing a second order Butterworth filter with the cutt-offs set usuallybetween 3-30 Hz (the time scale at which phonetic events occur in ratechanged speech). The band pass filtered envelope is then rectified toform the new envelope as follows:

e _(n) ^(new)(n)=S(e _(n)(n)*g(n))

where

S(x)=x for x>=0, otherwise S(x)=0

and g(n) is the impulse-response of the band-pass second orderButterworth filter. Flow then proceeds to block 3208.

At block 3208, the signal is modified within each band-pass channel tocarry this new envelope, as shown below:${f_{n}^{new}(n)} = {\left\lbrack {{f_{n}(n)}{S\left( \frac{e_{n}^{new}(n)}{e_{n}(n)} \right)}} \right\rbrack*{h(n)}}$

Flow then proceeds to block 3210.

At block 3210 the modified signal is obtained by summing the narrow-bandfilters with a differential gain for each channel as follows:${f^{new}(n)} = {\sum\limits_{n}{w_{n}{f_{n}^{new}(n)}}}$

where w_(n) is the gain for each channel. The envelope is modified onlywithin a specified frequency range from 1-10 KHz which normally spansabout 16 channels. Flow then proceeds to block 3212.

At block 3212 segmented digital speech output is provided.

Overlap-add Emphasis Algorithm

Referring to FIG. 33, a flow chart 3300 for an alternative emphasisalgorithm is provided. This algorithm improves upon the filter-banksummation described above by making use of the property of equivalencebetween the short-time Fourier transform and the filter-bank summationalgorithm. In this embodiment, the short-time Fourier transform iscomputed using an overlap-add procedure and the FFT algorithm. Flowbegins at block 3302 and proceeds to block 3304.

At block 3304, the short-time Fourier transform is computed over asliding window given by the following equation:${X_{k}(r)} = {\sum\limits_{m = {- \infty}}^{\infty}{{h\left( {r - n} \right)}{x(n)}^{{- {2}}\quad \pi \quad n\quad {k/N}}}}$

where h(n) is a Hamming window and the overlap between sections waschosen to be less than a quarter the length of the analysis window. Theenvelope can then be obtained within narrow-band channels from theabsolute value of the short-time Fourier transform. The number ofnarrow-band channels is equal to half the size of the length over whichthe FFT is computed.

The energy of the envelope within critical band channels is thenaveraged, as shown:${f_{n}(r)} = {\sum\limits_{C_{n - 1} \leq k \leq C_{n}}{{X_{k}(r)}}}$

where C_(n) is the corner-frequency of the critical-band channel n. Atpresent, critical-band frequencies for children with LLI are unknown,therefore the present invention approximates the bands using parametersproposed by Zwicker. See E. Zwicker and E. Terhardt, “Analyticalexpressions for critical-band rate and critical bandwidth as a functionof frequency,” J. Acoust. Soc. Ame., vol. 68, pp. 1523-25, 1980. Ascritical band frequencies for children with LLI become available, theycan be incorporated into the present invention.

The envelope within each critical-band channel is thenband-pass-filtered with cut off's set usually between 3-30 Hz with typeI linear phase FIR equiripple filters. The band-pass filtered envelopeis then threshold rectified. In contrast to the filter-bank emphasisalgorithm, the modified envelope is added to the original envelope toamplify the fast elements while not distorting the slower modulations.This is given by the following equation:${X_{k}^{new}(n)} = \left\lbrack {{X_{k}(n)}{T\left( \frac{e_{n}^{new}(n)}{e_{n}(n)} \right)}} \right\rbrack$

where,

T(x)=x+1 for x>=0, otherwise 0

Flow then proceeds to block 3308.

At block 3308, a modified signal is obtained by summing the short-timeFourier transform using a weighted overlap-add procedure as shown below:${f^{new}(n)} = {\sum\limits_{s = {- \infty}}^{\infty}{{g\left( {n - s} \right)}\quad \frac{1}{N}{\sum\limits_{k = 0}^{N - 1}{{X_{k}^{new}(n)}^{\quad 2\quad \pi \quad {{nk}/N}}}}}}$

where g(n) is the synthesis filter which was also chosen to be a Hammingwindow. Flow then proceeds to block 3310. At block 3310, windowing andover-lap addition for the algorithm is performed. Flow then proceeds toblock 3312 where segmented digital speech output is provided.

Although the present invention and its objects, features, and advantageshave been described in detail, other embodiments are encompassed by theinvention. For example, a number of different games have been shown,each dealing with a stimulus set that is processed in the time domain,and presented to a subject in a manner that the subject can understand.The processing is designed to emphasize or stretch those components ofspeech that are the most difficult for an LLI subject to differentiate,so that they may be more easily understood. In addition, the processingallows distinct frequency sweeps, phonemes or words to be separated intime, first by a significant amount, say 500 ms, with the amount ofseparation being gradually reduced to that of normal speech. Once asubject gains success at distinguishing between similar speech elements,at a high level of processing, the amount of processing is graduallyreduced until it reaches the level of normal speech. The particulargames used to train subjects have distinct advantages, but are notexclusive. Other games are anticipated that will incorporate the novelaspects of the present invention, to further train a subject's temporalprocessing ability to recognize and distinguish between short durationacoustic events that are common in speech.

Furthermore, the Fast ForWord program has been shown for execution on apersonal computer, connected to a central server. However, as technologyadvances, it is envisioned that the program could be executed either bya diskless computer attached to a server, by a handheld processingdevice, such as a laptop, or eventually by a palmtop device such as aNintendo GameBoy. As long as the graphical images and auditory promptscan be presented in a timely fashion, and with high quality, the natureof the device used to present the material is irrelevant.

Those skilled in the art should appreciate that they can readily use thedisclosed conception and specific embodiments as a basis for designingor modifying other structures for carrying out the same purposes of thepresent invention without departing from the spirit and scope of theinvention as defined by the appended claims.

Appendix A

Level 2

1. Attribute/Stative

Stimulus: The ball is big

The man is big.

The ball is small.

The ball is big.

Stimulus: The cup is broken

The wind up toy car is broken.

The cup is broken.

The cup is not broken.

Stimulus: The baby is crying

The baby is not crying.

The boy is crying.

The baby is crying.

Stimulus: The box is open

The box is open.

The box is closed.

The can is open.

Stimulus: The girl is dirty

The girl is dirty

The shoe is dirty.

The shoe is not dirty.

2. Simple Negation

Stimulus: The boy is not eating

The boy is not eating.

The boy is eating.

Stimulus: The boy is not riding

The boy is not riding.

The boy is riding.

Stimulus: The baby is not crying

The baby is crying.

The baby is not crying.

Stimulus: The boy does not have a balloon

The boy has a balloon.

The boy does not have a balloon.

Stimulus: The girl does not have shoes

The girl does not have shoes.

The girl has shoes.

3. Object Pronouns: Him&Her

Stimulus: Point to her.

A girl doll.

A boy doll.

Stimulus: Point to him

A girl doll.

A boy doll.

Stimulus: Point to her.

A girl doll.

boy doll.

Stimulus: Point to him.

A girl doll.

A boy doll.

Stimulus: Point to her.

A girl doll.

A boy doll.

4. Possession

Stimulus: The clown has a balloon.

The clown has a balloon

The clown has a flower.

The boy has a balloon.

Stimulus: The dog has spots.

The dog does not have spots.

The boy has spots.

The dog has spots.

Stimulus: The tree has apples.

The girl has apples.

The tree has apples.

The tree does not have apples.

Stimulus: The bunny has a carrot.

The bunny does not have a carrot.

The cat has a carrot.

The bunny has a carrot.

Stimulus: The girl has shoes.

The girl has shoes.

The girl does not have shoes.

The clown has shoes.

5. Lexicon Descriptions

Stimulus: Point to dark.

Dirty example.

Down example.

Dark example.

Big example.

Stimulus: Point to big.

Blue example.

Big example.

Open example.

Small example.

Stimulus: Point to dirty.

Dirty example.

Dark example.

Hot example.

Down example.

Stimulus: Point to yellow

Yellow example.

Small example

Broken example.

Blue example.

Stimulus: Point to off.

On example.

Open example.

Off example.

Wet example.

6. Lexicon Action Words

Stimulus: Point to wash.

Sweep example.

Fall example.

Wash example.

Run example.

Stimulus: Point to run.

Sing example.

Jump example.

Write example.

Run example.

Stimulus: Point to kick.

Throw example.

Kick example.

Push example.

Cry example.

Stimulus: Point to drink.

Eat example.

Run example.

Drink example.

Tear example.

Stimulus: Point to pull.

Pull example.

Push example.

Throw example.

Play example.

Level 3

b 7. Noun Singular/Plural (By Inflection Only)I

Stimulus: Point to the picture of the cups.

Point to the picture of the cup.

Point to the picture of the cups.

Stimulus: Point to the picture of the boat.

Point to the picture of the boat.

Point to the picture of the boats.

Stimulus: Point to the picture of the balloons.

Point to the picture of the balloons.

Point to the picture of the balloon.

Stimulus: Point to the picture of the houses.

Point to the picture of the house.

Point to the picture of the houses.

Stimulus: Point to the picture of the cat.

Point to the picture of the cat.

Point to the picture of the cats.

8. Qualifiers: None

Stimulus: Which clown has none?

The clown has many.

The clown has some.

The clown has one.

The clown has none.

Stimulus: Which tree has none?

The tree has none.

The tree has one.

The tree has some.

The tree has many.

Stimulus: Which dog has none?

The dog has some.

The dog has many.

The dog has none.

The dog has one.

Stimulus: Which duck has none?

The duck has many.

The duck has none.

The duck has one.

The duck has some.

Stimulus: Which wagon has none?

The wagon has some.

The wagon has none.

The wagon has many.

The wagon has one.

9. Subject Relativization

Stimulus: The boy who is mad is pulling the girl.

The boy who is happy is pulling the girl.

The boy who is mad is pulling the girl.

The girl who is mad is pulling the boy.

The girl who is happy is pulling the boy.

Stimulus: The girl who is mad is pushing the boy.

The girl who is happy is pushing the boy.

The boy who is mad is pushing the girl.

The girl who is mad is pushing the boy.

The boy who is happy is pushing the girl.

Stimulus: The clown who is big is chasing the girl.

The clown who is big is chasing the girl.

The girl who is little is chasing the clown.

The girl who is big is chasing the clown.

The clown who is little is chasing the girl.

Stimulus: The girl who is happy is pulling the boy.

The boy who is happy is pulling the girl.

The girl who is mad is pulling the boy.

The boy who is mad is pulling the girl.

The girl who is happy is pulling the boy.

Stimulus: The clown who is little is chasing the girl.

The girl who is little is chasing the clown.

The clown who is big is chasing the girl.

The girl who is big is chasing the clown.

The clown who is little is chasing the girl.

10. Active Voice Word Order

Stimulus: The boy is pulling the girl.

The boy is pulling the ball.

The girl is pulling the boy.

The boy and the girl are pulling the wagon.

The boy is pulling the girl.

Stimulus: The boy is pushing the girl.

The girl is pushing the boy.

The boy is pushing the girl.

The boy is pushing the clown.

They are pushing the ball.

Stimulus: The girl is pulling the boy.

The girl is pulling the boy.

They are pulling the wagon.

The dog is pulling the boy.

The boy is pulling the girl.

Stimulus: The boy is kicking the girl.

They are kicking the balls.

The girl is kicking the boy.

The clown is kicking the girl.

The boy is kicking the girl.

Stimulus: The girl is washing the boy.

They are washing the tub.

The girl is washing the dog.

The girl is washing the boy.

The boy is washing the girl.

11. Comparative with More

Stimulus: Which one is more happy?

Which one is less happy?

Which one is more happy?

Stimulus: Which one is more hairy?

Which one is more hairy?

Which one is less hairy?

Stimulus: Which one is more skinny?

Which one is less skinny?

Which one is more skinny?

Stimulus: Which one is more dirty?

Which one is more dirty?

Which one is less dirty?

Stimulus: Which one is more messy?

Which one is more messy?

Which one is less messy?

12. Reduced Subject Relative Clauses

Stimulus: The boy frowning is pulling the girl.

The girl frowning is pulling the boy.

The boy smiling is pulling the girl.

The girl smiling is pulling the boy.

The boy frowning is pulling the girl.

Stimulus: The boy smiling is pushing the girl.

The girl frowning is pushing the boy.

The boy smiling is pushing the girl.

The girl smiling is pushing the boy.

The boy frowning is pushing the girl.

Stimulus: The boy frowning is pushing the girl.

The boy frowning is pushing the girl.

The girl smiling is pushing the boy.

The boy smiling is pushing the girl.

The girl frowning is pushing the boy.

Stimulus: The girl smiling is pushing the boy.

The boy smiling is pushing the girl.

The girl frowning is pushing the boy.

The girl smiling is pushing the boy.

The boy frowning is pushing the girl.

Stimulus: The girl frowning is pulling the boy.

The girl frowning is pulling the boy.

The boy frowning is pulling the girl.

The girl smiling is pulling the boy.

The boy smiling is pulling the girl.

13. Complex Negation

Stimulus: The clown that is not on the box is little.

The clown that is not on the box is little.

The clown that is on the box is little.

The clown that is not on the box is big.

The clown that is on the box is big.

Stimulus: The girl that is chasing the clown is not big

The clown that is chasing the girl is big.

The girl that is chasing the clown is not big.

The clown that is chasing the girl is not big.

The girl that is chasing the clown is big.

Stimulus: The boy that is not sitting is looking at the girl.

The girl that is not sitting is not looking at the boy.

The girl that is not sitting is looking at the boy

The boy that is not sitting is looking at the girl.

The boy that is not sitting is not looking at the girl.

Stimulus: The clown that is big is not on the box.

The clown that is little is not on the box.

The clown that is big is on the box.

The clown that is little is on the box.

The clown that is big is not on the box.

Stimulus: The book that is not on the table is blue.

The book that is on the table is blue.

The book that is on the table is red.

The book that is not on the table is blue.

The book that is not on the table is red.

14. Noun Singular/Plural (By Inflection Only)

Stimulus: Point to the picture of the watches.

Point to the picture of the watch.

Point to the picture of the watches.

Stimulus: Point to the picture of the tubs.

Point to the picture of the tub.

Point to the picture of the tubs.

Stimulus: Point to the picture of the hat.

Point to the picture of the hats.

Point to the picture of the hat.

Stimulus: Point to the picture of the bunny.

Point to the picture of the bunnies.

Point to the picture of the bunny.

Stimulus: Point to the picture of the socks.

Point to the picture of the sock.

Point to the picture of the socks.

15. Comparative with/ -er/

Stimulus: Which one is happier?

Which one is happier?

Which one is happy?

Stimulus: Which one is hairier?

Which one is hairier?

Which one is more bald?

Stimulus: Which one is skinnier?

Which one is heavier?

Which one is skinnier?

Stimulus: Which one is dirtier?

Which one is dirtier?

Which one is cleaner?

Stimulus: Which one is messier?

Which one is messier?

Which one is cleaner?

Level 4

16. Passive Voice Word Order

Stimulus: The boy is being pushed by the girl.

The boy is being pushed by the clown.

The clown is being pushed by the girl.

The girl is being pushed by the boy.

The boy is being pushed by the girl.

Stimulus: The dog is being pulled by the clown.

The dog is being pulled by the boy.

The clown is being pulled by the dog.

The dog is being pulled by the clown.

The girl is being pulled by the clown.

Stimulus: The girl is being kicked by the boy.

The girl is being kicked by the boy.

The clown is being kicked by the boy.

The girl is being kicked by the clown.

The boy is being kicked by the girl.

Stimulus: The boy is being chased by the girl.

The dog is being chased by the girl.

The girl is being chased by the boy.

The boy is being chased by the girl.

The boy is being chased by the dog.

Stimulus: The boy is being kicked by the girl.

The girl is being kicked by the boy.

The boy is being kicked by the girl.

The clown is being kicked by the girl.

The boy is being kicked by the clown.

17. Wh-Object Questioning

Stimulus: What is the cat chasing? (the mouse)

What is the dog chasing? (the cat)

What is the cat chasing? (the mouse)

What is the mouse chasing? (the cat)

Stimulus: Who is the clown chasing? (the boy)

Who is the girl chasing?

Who is the clown chasing?

Who is the boy chasing?

Stimulus: Who is the girl pushing? (the boy)

Who is the mother pushing?

Who is the girl pushing?

Who is the boy pushing?

Stimulus: Who is the boy pulling? (the girl)

Who is the girl pulling?

Who is the boy pulling?

Who is the clown pulling?

Stimulus: Who is the girl pulling? (the clown)

Who is the clown pulling?

Who is the girl pulling?

Who is the boy pulling?

18. Quantifiers: Some

Stimulus: Look at these wagons with deer.

Which wagon has some?

Which wagon has one?

Which wagon has some?

Which wagon has many?

Which wagon has none?

Stimulus: Look at these clown with balloons.

Which clown has some?

Which clown has one?

Which clown has none?

Which clown has some?

Which clown has many?

Stimulus: Look at these ducks with babies?

Which duck has some?

Which duck has none?

Which duck has many?

Which duck has some?

Which duck has one?

Stimulus: Look at these trees with apples.

Which tree has some?

Which tree has one?

Which tree has none?

Which tree has many?

Which tree has some?

Stimulus: Look at these dogs with spots.

Which dog has some?

Which dog has some?

Which dog has none?

Which dog has one?

Which dog has many?

19. Verb Singular

Stimulus: The fish swims.

The fish swims.

The fish swim.

Stimulus The sheep stands.

The sheep stand.

The sheep stands.

Stimulus: The deer drinks.

The deer drinks.

The deer drink.

Stimulus: The fish eats.

The fish eat.

The fish eats.

Stimulus: The sheep jumps.

The sheep jumps.

The sheep jump.

20. Tense and Aspect: ing

Stimulus: The girl is opening the present.

The girl opened the present.

The girl is opening the present.

The girl will open the present.

Stimulus: The boy is washing his face.

The boy is washing his face.

The boy will wash his face.

The boy washed his face.

Stimulus: The boy is pouring the juice.

The boy will pour the juice.

The boy poured the juice.

The boy is pouring the juice.

Stimulus: The girl is blowing up the balloon.

The girl is blowing up the balloon.

The girl will blow up the balloon.

The girl blew up the balloon.

Stimulus: The boy is eating his dinner.

The boy will eat his dinner.

The boy ate his dinner.

The boy is eating his dinner.

21. Noun Plurals/Singulars Marked by Quantifier

Stimulus: Point to the picture of some socks.

Point to the picture of a sock.

Point to the picture of some socks.

Stimulus: Point to the picture of a bag.

Point to the picture of a bag.

Point to the picture of some bags.

Stimulus: Point to the picture of some dresses.

Point to the picture of some dresses.

Point to the picture of a dress.

Stimulus: Point to the picture of a bunny.

Point to the picture of a bunny.

Point to the picture of some bunnies.

Stimulus: Point to the picture of some dogs.

Point to the picture of a dog.

Point to the picture of some dogs.

40. Noun Plurals/Singulars Marked by Quantifier Inflection

Note # above is for programatic purposes only. Level is correct.

Stimulus: Point to the picture of some balloons.

Point to the picture of a balloon.

Point to the picture of some balloons.

Stimulus: Point to the picture of a tree.

Point to the picture of some trees.

Point to the picture of a tree.

Stimulus: Point to the picture of some cats.

Point to the picture of some cats.

Point to the picture of a cat.

Stimulus: Point to the picture of some boxes.

Point to the picture of some boxes.

Point to the picture of a box.

Stimulus: Point to the picture of a cake.

Point to the picture of a cake.

Point to the picture of some cakes.

22. Aux-Be Singular

Stimulus: The fish is swimming

The fish is swimming.

The fish are swimming.

Stimulus: The sheep is standing.

The sheep are standing.

The sheep is standing.

Stimulus: The deer is drinking.

The deer is drinking.

The deer are drinking.

Stimulus: The fish is eating.

The fish are eating.

The fish is eating.

Stimulus: The sheep is jumping.

The sheep is jumping.

The sheep are jumping.

Level 5

23. Case Marking Prepositions: For

Stimulus: Show me the groceries being carried for mom

Show me the groceries being carried for mom

Show me the groceries being carried with mom

Show me the groceries being carried by mom.

Stimulus: Show me the breakfast made for mom.

Show me the breakfast made by mom.

Show me the breakfast made for mom.

Show me the breakfast made with mom.

Stimulus: Show me the drawing made for the boy.

Show me the drawing of the boy.

Show me the drawing made by the boy.

Show me the drawing made for the boy.

Stimulus: Show me the suitcase being carried for the man

Show me the suitcase being carried with the man.

Show me the suitcase being carried for the man.

Show me the suitcase being carried by the man.

Stimulus: Show me the painting made for the girl

Show me the painting made for the girl.

Show me the painting made by the girl.

Show me the painting made of the girl.

24. Tense and Aspect: -ed

Stimulus: The girl painted a picture.

The girl painted a picture.

The girl will paint a picture.

The girl is painting a picture.

Stimulus: The man sewed a shirt.

The man will sew a shirt.

The man is sewing a shirt.

The man sewed a shirt.

Stimulus: Someone tied the shoe.

Someone is tying the shoe.

Someone tied the shoe.

Someone will tie the shoe.

Stimulus: The boy tripped over the rock.

The boy tripped over the rock.

The boy is tripping over the rock.

The boy will trip over the rock.

Stimulus: The mother dressed the baby.

The mother is dressing the baby.

The mother will dress the baby.

The mother dressed the baby.

25. Aux-Be Plural

Stimulus: The deer are eating.

The deer is eating.

The deer are eating

Stimulus: The sheep are jumping.

The sheep are jumping.

The sheep is jumping.

Stimulus: The fish are swimming.

The fish is swimming.

The fish are swimming.

Stimulus: The deer are standing.

The deer is standing.

The deer are standing.

Stimulus: The sheep are eating.

The sheep are eating.

The sheep is eating.

26. Third Person Subject Pronouns

Stimulus: Point to they are sitting.

Point to she is sitting.

Point to they are sitting.

Point to he is sitting.

Stimulus: Point to she is jumping.

Point to they are jumping.

Point to he is jumping.

Point to she is jumping.

Stimulus: Point to he is standing.

Point to he is standing.

Point to they are standing.

Point to she is standing.

Stimulus: Point to she is kicking.

Point to they are kicking.

Point to she is kicking.

Point to he is kicking.

Stimulus: Point to they are eating.

Point to he is eating.

Point to she is eating.

Point to they are eating.

Level 6

27. Tense and Aspect: will

Stimulus: The girl will open the present.

The girl opened the present.

The girl is opening the present.

The girl will open the present.

Stimulus: The boy will eat his dinner.

The boy ate his dinner.

The boy will eat his dinner.

The boy is eating his dinner.

Stimulus: The boy will trip over the rock.

The boy will trip over the rock.

The boy is tripping over the rock.

The boy tripped over the rock.

Stimulus: The man will sew his shirt.

The man is sewing his shirt.

The man sewed his shirt.

The man will sew his shirt.

Stimulus: The girl will paint a picture.

The girl is painting a picture.

The girl will paint a picture.

The girl painted a picture.

28. Possessive Morpheme /′s/

Stimulus: Show me the baby bear.

Show me the baby's bear.

Show me the baby bear.

Stimulus: Show me the chicken's dinner.

Show me the chicken's dinner.

Show me the chicken dinner.

Stimulus: Show me the mama cat.

Show me the mama cat.

Show me the mama's cat.

Stimulus: Show me the baby's duck.

Show me the baby duck.

Show me the baby's duck.

Stimulus: Show me the baby's bunny.

Show me the baby bunny.

Show me the baby's bunny.

29. Case Marking Prepositions: With

Stimulus: Show me the breakfast made with mom.

Show me the breakfast made for mom.

Show me the breakfast made by mom.

Show me the breakfast made with mom.

Stimulus: Show me the suitcase being carried with the man.

Show me the suitcase being carried by the man

Show me the suitcase being carried with the man.

Show me the suitcase being carried for the man.

Stimulus: Show me the baby walking with the girl.

Show me the baby walking with the girl.

Show me the baby walking to the girl.

Show me the baby walking from the girl.

Stimulus: Show me the boy running with the girl.

Show me the boy running with the girl.

Show me the boy running from the girl.

Show me the boy running to the girl.

Stimulus: Show me the groceries being carried with mom

Show me the groceries being carried for mom.

Show me the groceries being carried with mom.

Show me the groceries being carried by mom.

30. Double Embedding

Stimulus: The clown that is chasing the girl that is little is big.

The clown that is chasing the girl that is big is little.

The clown that is chasing the girl that is little is little.

The clown that is chasing the girl that is big is big.

The clown that is chasing the girl that is little is big.

Stimulus: The clown that is holding the balloon that is red is blue.

The clown that is holding the balloon that is blue is blue.

The clown that is holding the balloon that is red is blue.

The clown that is holding the balloon that is red is red.

The clown that is holding the balloon that is blue is red.

Stimulus: The girl that is chasing the clown that is big is little.

The girl that is chasing the clown that is big is big.

The girl that is chasing the clown that is little is big.

The girl that is chasing the clown that is big is little.

The girl that is chasing the clown that is little is little.

Stimulus: The clown that is holding the balloon that blue is red.

The clown that is holding the balloon that is blue is red.

The clown that is holding the balloon that is red is red.

The clown that is holding the balloon that is red is blue.

The clown that is holding the balloon that is blue is blue.

Stimulus: The girl that is chasing the clown that is little is big.

The girl that is chasing the clown that is big is big.

The girl that is chasing the clown that is little is little.

The girl that is chasing the clown that is little is big.

The girl that is chasing the clown that is big is little.

31. Relativized Subject Ending in N-V-N

Stimulus: The girl who is pushing the boy is happy

The boy who is pushing the girl is mad.

The girl who is pushing the boy is mad.

The girl who is pushing the boy is happy.

The boy who is pushing the girl is happy.

Stimulus: The clown who is chasing the girl is little

The clown who is chasing the girl is little.

The girl who is chasing the clown is little.

The girl who is chasing the clown is big.

The clown who is chasing the clown is big.

Stimulus: The boy who is pulling the girl is mad.

The girl who is pulling the boy is happy.

The boy who is pulling the girl is mad.

The boy who is pulling the girl is happy.

The girl who is pulling the boy is mad.

Stimulus: The girl who is chasing the clown is little.

The clown who is chasing the girl is big.

The girl who is chasing the clown is big.

The girl who is chasing the clown is little.

The clown who is chasing the girl is little.

Level 7

32. Object Relativization

Stimulus: The girl is chasing the clown who is big.

The clown is chasing the girl who is little.

The girl is chasing the clown who is big.

The clown is chasing the girl who is big.

The girl is chasing the clown who is little.

Stimulus: The boy is pushing the girl who is happy.

The boy is pushing the girl who is happy.

The girl is pushing the boy who is happy.

The girl is pushing the boy who is mad.

The boy is pushing the girl who is mad.

Stimulus: The girl is pulling the boy who is mad

The boy is pulling the girl who is mad.

The boy is pulling the girl who is happy.

The girl is pulling the boy who is happy.

The girl is pulling the boy who is mad.

Stimulus: The boy is pushing the girl who is mad

The boy is pushing the girl who is mad.

The girl is pushing the boy who is mad.

The girl is pushing the boy who is happy.

The boy is pushing the girl who is happy.

Stimulus: The girl is chasing the clown who is little.

The clown is chasing the girl who is little.

The girl is chasing the clown who is big.

The girl is chasing the clown who is little.

The clown is chasing the girl who is big.

33. Reduced Subject Relative Clauses Ending in -V-N

Stimulus: The girl pushing the boy is smiling

The girl pushing the boy is smiling.

The boy pushing the girl is smiling.

The girl pushing the boy is frowning.

The boy pushing the girl is frowning.

Stimulus: The clown chasing the girl is little

The clown chasing the girl is big.

The girl chasing the clown is big.

The clown chasing the girl is little.

The girl chasing the clown is little.

Stimulus: The girl pulling the boy is frowning.

The girl pulling the boy is smiling.

The boy pulling the girl boy is frowning.

The boy pulling the girl is smiling.

The girl pulling the boy is frowning.

Stimulus: The girl chasing the clown is little.

The clown chasing the girl is little.

The girl chasing the clown is little.

The clown chasing the girl is big.

The girl chasing the clown is big.

Stimulus: The boy pulling the girl is frowning

The girl pulling the boy is smiling.

The boy pulling the girl is frowning.

The girl pulling the boy is frowning.

The boy pulling the girl is smiling.

34. Who vs. What

Stimulus: What is in the wagon? (ball)

Is nothing in the wagon?

Who is in the wagon?

What is in the wagon? (ball)

Stimulus: Who is in the tub? (man)

What is in the tub?

Who is in the tub? (man)

Is nothing in the tub?

Stimulus: What is under the table (cup)

Who is under the table?

What is under the table?

Is nothing under the table?

Stimulus: What is on the chair? (ball)

What is on the chair? (ball)

Is nothing on the chair?

Who is on the chair?

Stimulus: Who is on the box (clown)

Who is on the box?

What is on the box?

Is nothing on the box?

Level 8

35. Verb Plural

Stimulus: The deer eat.

The dear eats.

The deer eat.

Stimulus: The sheep jump.

The sheep jump.

The sheep jumps

Stimulus: The fish swim.

The fish swims.

The fish swim.

Stimulus: The deer stand.

The deer stands.

The deer stand.

Stimulus: The sheep eat.

The sheep eat.

The sheep eats.

36. Relative Pronouns with Double Function

Stimulus: The girl who the boy girl who the boy is pushing is happy

The girl who the boy is pushing is mad.

The boy who the girl is pushing is mad.

The girl who the boy is pushing is happy.

The boy who the girl is pushing is happy.

Stimulus: The boy who the girl is pulling is mad.

The boy who the girl is pulling is mad.

The boy who the girl is pulling is happy.

The girl who the boy is pulling is happy.

The girl who the boy is pulling is mad.

Stimulus: The girl who the clown is chasing, is chasing the boy.

The clown who the boy is chasing, is chasing the girl.

The girl who the clown is chasing, is chasing the boy.

The clown who the girl is chasing, is chasing the boy.

Stimulus The boy who the girl is pulling, is pulling the clown

The girl who the boy is pulling, is pulling the clown.

The clown who the boy is pulling, is pulling the girl.

The clown who the girl is pulling, is pulling the boy.

The boy who the girl is pulling, is pulling the clown.

Stimulus The boy who the girl is pushing, is happy.

The boy who the girl is pushing is happy.

The girl who the boy is pushing is sad.

The boy who the girl is pushing is mad.

The girl who the boy is pushing is happy.

37. Object Relatives with Relativized Objects

Stimulus The girl is hugging the boy that the clown is kissing.

The girl is hugging the boy that is kissing

The boy is hugging the girl that the clown

The girl is hugging the boy that the clown

The girl is hugging the clown that the boy

Stimulus The clown is hugging the girl that the boy is kissing

The clown is hugging the boy that the girl is kissing.

The clown is hugging the girl that the boy is kissing.

The clown is hugging the girl that is kissing the boy.

The girl is hugging the clown that the boy is kissing.

Stimulus The girl is kissing the clown that the boy is hugging

The girl is kissing the boy that the clown is hugging.

The clown is kissing the girl that the boy is hugging.

The girl is kissing the clown that the boy is hugging.

The girl is kissing the clown that is hugging the boy.

Stimulus The boy is hugging the girl that the clown is kissing

The girl is hugging the boy that the clown is kissing.

The boy is hugging the girl that is kissing the clown.

The boy is hugging the clown that the girl is kissing.

The boy is hugging the girl that the clown is kissing.

Stimulus The boy is kissing the girl that the clown is hugging

The boy is kissing the girl that is hugging the clown.

The girl is kissing the boy that the clown is hugging.

The boy is kissing the girl that the clown is hugging.

The boy is kissing the clown that the girl is hugging.

38. Clefting

Stimulus: It's the clown that the girl chases.

It's the girl that the clown chases.

It's the boy that the clown chases.

It's the boy that the girl chases.

It's the clown that the girl chases.

Stimulus: It's the boy that the girl kicks.

It's the clown that the boy kicks.

It's the clown that the girl kicks.

It's the boy that the girl kicks.

It's the girl that the boy kicks.

Stimulus: It's the girl that the boy pulls.

It's the clown that the boy pulls.

It's the girl that the boy pulls.

It's the boy that the girl pulls.

It's the clown that the girl pulls.

Stimulus: It's the boy that the clown pushes.

It's the boy that the clown pushes.

It's the girl that the clown pushes.

It's the girl that the boy pushes.

It's the clown that the boy pushes.

Stimulus: It's the boy that the clown chases.

It's the girl that the boy chases.

It's the clown that the boy chases.

It's the boy that the clown chases.

It's the girl that the clown chases.

39. Negative-Passive

Stimulus The cat is not being outrun by the dog.

The cat is not being outrun by the dog.

The dog is not being outrun by the cat.

The boy is not being outrun by the cat.

Stimulus The drawing is not to be received by the girl

The drawing is not to be received by the girl.

The drawing is not to be received by the woman.

The drawing is not to be received by the girl.

Stimulus The boy is not followed by the girl.

The girl is not followed by the boy.

The boy is not followed by the girl.

The clown is not followed by the boy.

Stimulus The picture is not to be received from the boy

The picture is not to be received from the boy.

The picture is not to be received from the man.

The picture is not to be received from the girl.

Stimulus The clown is not led by the girl.

The clown is not led by the boy.

The girl is not led by the clown.

The clown is not led by the girl

We claim:
 1. An adaptive method for improving a subject's short termmemory, the method utilizing a computer to process and present sounds tothe subject, the method comprising: a) displaying at least fourgraphical images, the graphical images associated with processedacoustic events, the acoustic events being consonant-vowel combinations;b) associating in pairs the at least four graphical images withparticular acoustic events such that each pair of different graphicalimages arc associated with the same acoustic event; c) upon selection ofany of the at least four graphical images, presenting its associatedacoustic event; and d) requiring the user to discriminate between theacoustic events by sequentially selecting the pair of differentgraphical images from among the at least four graphical images, that areassociated with the same acoustic event; e) when the user sequentiallyselects the pair of different graphical images that are associated withthe same acoustic event, from the at least four graphical images,removing the pair of different graphical images that are associated withthe same acoustic event from game play, and repeating said steps a)-e)until all of the at least four graphical images have been removed. 2.The adaptive method as recited in claim 1 wherein the at least fourgraphical images comprise an even number of images such that every twographical images are associated with a same acoustic event.
 3. Theadaptive method as recited in claim 1 wherein the acoustic events areshort duration acoustic events, typically between 35 ms and 150 ms, thatare modified by stretching in the time domain, without significantlyeffecting their associated frequency components.
 4. The adaptive methodas recited in claim 3 wherein each of the acoustic events are stretchedin the time domain into a plurality of modified acoustic events that arestretched between 100% and 150%.
 5. The adaptive method as recited inclaim 3 wherein the modified acoustic events have selected frequencyenvelopes that are emphasized, by increasing in amplitude the selectedfrequency envelopes, to ease discrimination between acoustically similarmodified acoustic events.
 6. The adaptive method as recited in claim 3further comprising: f) presenting a plurality of skill levels to thesubject, the skill levels increasing in difficulty from approximately150% stretching and 20 dB emphasis of a particular modified acousticevent, to no stretching and no emphasis (normal speech).
 7. The adaptivemethod as recited in claim 6 further comprising: g) as the usercorrectly discriminates between the acoustic events by sequentiallyselecting the pair of different graphical images from among the at leastfour graphical images, a predetermined number of times, increasing theskill level presented to the user.